From slander at unworkable.org Tue Feb 2 19:54:09 2010 From: slander at unworkable.org (Harry Tormey) Date: Tue, 2 Feb 2010 10:54:09 -0800 Subject: [Baypiggies] PyGameSF meetup Wednesday Febrauary 17th 6pm @ Stong room Main San Francisco Public Library Message-ID: <20100202185409.GA13906@unworkable.org> Hi All, The February PyGameSF meet up will be at the STONG conference room on the first floor of the main San Francisco public library beside civic center BART. The library closes at 8pm so we will reconvene to frjtz on hayes street for dinner/drinks afterwords. This month's presentations are: * Shandy Brown : Structuring Your Game's Code. One approach to designing video games with a focus on rapid development and networked multiplayer capabilities. Shandy Brown will highlight key ideas in his tutorial (http://ezide.com/games/writing-games.html). Fundamental topics such as event-based design, defining your game model, and separating the model and the view will be covered. * Warren Stringer Tr3: A Tablet Ontology. Warren will be demonstrating Tr3 (pronounced tree), a media performance ontology that connects Human gestures to device interfaces, such as the iPad. Tr3 was designed for the novice, programmer, and designer. A novice will use Tr3 to orchestrate interactive media. A programmer will use Tr3 to wrap real-time APIs around their preferred language. A designer will use Tr3 to iterate through many user interfaces. Tr3 was started in 1998, as a patch bay for a synthesizer that used a variety of input devices to create visual music. The favorite device is a tablet, which was put in the hands of all age groups with universally mesmerizing results. Tr3 has been re-written to focus on multi-touch surfaces. Tr3 consists of a namespace, auto-scaled types, tree decorators, event observers, recordable sessions, code wrapper, ontology viewer, and a plug-in architecture for selling performances. An overview can be ound here: http://muse.com/tr3 Since this is a PyGame group, Warren will be demonstrating how Python reads in theTr3 definition and spits out a language parser in either Python or C++. Depending on availability, Warren will demonstrate a Tr3 wrapping a tablet based visual music synthesizer. PyGame SF is an informal group meet up in San Francisco for Software engineers interested in python, OpenGL, audio, pygame, SDL, programming and generally anything to do with multimedia development. The format of our meetings typically involve several people giving presentations on projects they are developing followed by group discussion and feedback. If anyone else would like to give a micro presentation, show demos or just talk about what they are doing or generally give examples of any relevant software they are working on please feel free to head along To subscribe to the pygamesf mailing list simply email: pygame-sf+subscribe at unworkable.org -- Harry Tormey Co Founder P2P Research http://p2presearch.com Founder PyGameSF http://pygamesf.org Software Engineer Digidesign http://digidesign.com From crshonda at hotmail.com Tue Feb 2 21:17:57 2010 From: crshonda at hotmail.com (Crshonda Williams) Date: Tue, 2 Feb 2010 12:17:57 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA Message-ID: Please send resumes to: crshonda at chompapps.com Crawler/Backend Engineer Chomp San Francisco, CA Overview Chomp is seeking an experienced backend engineer to take ownership of our crawling and search engine components. We're ideally looking for someone with experience at a search company, although this isn't essential. We're looking for a regex and scripting wizard with experience building multi-threaded applications. You'll also be in charge of our search indexing and ranking algorithms, so this is very much an all-rounder position. The right candidate will be excited by that rather than intimidated. Check us out here: www.chompapps.com Us: - Chomp is an exciting new mobile start-up, funded by top Silicon Valley investors & working on the biggest mobile problem the world faces today. - We are engineers, smart & aggressive... & have built and sold companies in the mobile space previously. You: - Will be working with an extremely small team & will get in on this opportunity early! - Smart, hungry and ready to make your mark. It would be a plus if you have a good sense of humor too. :-) - Prepared to do whatever it takes to succeed. - Passionate about building products that blow people away. Responsibilities - Take ownership of the search components of our backend infrastructure - in particular our crawler and ranking algorithm. - Write and maintain high quality, scalable, unit tested code. - Be involved in QA testing. Experience - Strong Python skills. Experience with building multi-threaded applications in Python - Strong MySQL skills, including both query and database design. - Backend experience at a search company is highly desired. - Experience with open source search engine libraries such as Lucene or Sphinx highly desired. - Experience with distributed version control, preferably Git. _________________________________________________________________ Hotmail: Trusted email with powerful SPAM protection. http://clk.atdmt.com/GBL/go/201469227/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tungwaiyip at yahoo.com Tue Feb 2 22:28:41 2010 From: tungwaiyip at yahoo.com (Tung Wai Yip) Date: Tue, 02 Feb 2010 13:28:41 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: References: Message-ID: On Tue, 02 Feb 2010 12:17:57 -0800, Crshonda Williams wrote: > > Please send resumes to: crshonda at chompapps.com > Crawler/Backend Engineer > Chomp > San Francisco, CA > .... > Experience > - Strong Python skills. Experience with building multi-threaded > applications in Python I think my experience tells me don't build multi-threaded applications in Python ;-) Wai Yip From aahz at pythoncraft.com Tue Feb 2 23:14:25 2010 From: aahz at pythoncraft.com (Aahz) Date: Tue, 2 Feb 2010 14:14:25 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: References: Message-ID: <20100202221425.GA7800@panix.com> On Tue, Feb 02, 2010, Tung Wai Yip wrote: > > I think my experience tells me don't build multi-threaded applications in > Python ;-) Why? I've had about equal trouble with multi-processing and multi-threading. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From langton2 at llnl.gov Tue Feb 2 23:39:19 2010 From: langton2 at llnl.gov (Asher Langton) Date: Tue, 2 Feb 2010 14:39:19 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <20100202221425.GA7800@panix.com> References: <20100202221425.GA7800@panix.com> Message-ID: <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> On Feb 2, 2010, at 2:14 PM, Aahz wrote: > On Tue, Feb 02, 2010, Tung Wai Yip wrote: >> >> I think my experience tells me don't build multi-threaded >> applications in >> Python ;-) > > Why? I've had about equal trouble with multi-processing and > multi-threading. If you haven't seen Dave Beazley's presentations on the Global Interpreter Lock, take a look at these: http://www.dabeaz.com/python/GIL.pdf http://www.dabeaz.com/python/NewGIL.pdf In short: multi-threading is really bad in Python 2.x when the threads are CPU-bound. -Asher From alecf at flett.org Tue Feb 2 23:50:00 2010 From: alecf at flett.org (Alec Flett) Date: Tue, 2 Feb 2010 14:50:00 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> References: <20100202221425.GA7800@panix.com> <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> Message-ID: Or even kind of CPU-bound, as I discovered some months back... lets say you have a process that would normally be about 50% blocked on IO, and 50% of the time chewing on the data retrieved from that IO blocking. You run more than 2-3 of those on a multi-processor box, and you start getting contention. desperately hoping some of the various things being tried in unladen-swallow pay off, Alec On Tue, Feb 2, 2010 at 2:39 PM, Asher Langton wrote: > On Feb 2, 2010, at 2:14 PM, Aahz wrote: > >> On Tue, Feb 02, 2010, Tung Wai Yip wrote: >> >>> >>> I think my experience tells me don't build multi-threaded applications in >>> Python ;-) >>> >> >> Why? I've had about equal trouble with multi-processing and >> multi-threading. >> > > If you haven't seen Dave Beazley's presentations on the Global Interpreter > Lock, take a look at these: > > http://www.dabeaz.com/python/GIL.pdf > http://www.dabeaz.com/python/NewGIL.pdf > > In short: multi-threading is really bad in Python 2.x when the threads are > CPU-bound. > > > -Asher > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From langton2 at llnl.gov Tue Feb 2 23:51:53 2010 From: langton2 at llnl.gov (Asher Langton) Date: Tue, 2 Feb 2010 14:51:53 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> References: <20100202221425.GA7800@panix.com> <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> Message-ID: <0D5563DC-9F94-46CA-8328-6551B57DF0FA@llnl.gov> On Feb 2, 2010, at 2:39 PM, Asher Langton wrote: > On Feb 2, 2010, at 2:14 PM, Aahz wrote: >> On Tue, Feb 02, 2010, Tung Wai Yip wrote: >>> >>> I think my experience tells me don't build multi-threaded >>> applications in >>> Python ;-) >> >> Why? I've had about equal trouble with multi-processing and >> multi-threading. > > If you haven't seen Dave Beazley's presentations on the Global > Interpreter Lock, take a look at these: > > http://*www.*dabeaz.com/python/GIL.pdf > http://*www.*dabeaz.com/python/NewGIL.pdf > > In short: multi-threading is really bad in Python 2.x when the > threads are CPU-bound. Oops, I just looked at your webpage, and saw that you _wrote_ an article on the GIL (though I can't access it right now), so you probably didn't need these links! Hopefully they're helpful to others, though. -Asher From wescpy at gmail.com Wed Feb 3 01:20:04 2010 From: wescpy at gmail.com (wesley chun) Date: Tue, 2 Feb 2010 16:20:04 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: References: <20100202221425.GA7800@panix.com> <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> Message-ID: <78b3a9581002021620g5ced22c7u22fb2309f6602620@mail.gmail.com> >> In short: multi-threading is really bad in Python 2.x when the threads are CPU-bound. some will even argue it *isn't* multithreading in this degenerative case... it's just round-robin. -- wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - "Python Web Development with Django", Addison Wesley, (c) 2009 http://withdjango.com wesley.j.chun :: wescpy-at-gmail.com python training and technical consulting cyberweb.consulting : silicon valley, ca http://cyberwebconsulting.com From glen at glenjarvis.com Wed Feb 3 02:13:04 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Tue, 2 Feb 2010 17:13:04 -0800 Subject: [Baypiggies] Number of inodes on a volume? Message-ID: I have a quick question to this brilliant group :) How do I, using pure python, get the number of inodes free on a volume for a particular mount? Basically, is there an equivalent to df -i in Python? Is this somewhere in stat and I can't find it? Cheers, Glen From aahz at pythoncraft.com Wed Feb 3 02:15:39 2010 From: aahz at pythoncraft.com (Aahz) Date: Tue, 2 Feb 2010 17:15:39 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> References: <20100202221425.GA7800@panix.com> <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> Message-ID: <20100203011538.GA11207@panix.com> On Tue, Feb 02, 2010, Asher Langton wrote: > On Feb 2, 2010, at 2:14 PM, Aahz wrote: >> On Tue, Feb 02, 2010, Tung Wai Yip wrote: >>> >>> I think my experience tells me don't build multi-threaded >>> applications in Python ;-) >> >> Why? I've had about equal trouble with multi-processing and >> multi-threading. > > In short: multi-threading is really bad in Python 2.x when the threads > are CPU-bound. Absolutely. Well, almost -- it's still really good for creating a responsive GUI application. See my Fibonacci example in my threads tutorial. Nevertheless, that's completely different from "don't use threads". PS: you can remove the "2.x" from your comment; even if the Unladen Swallow people improve GIL handling, it's almost certain that CPU-bound threads will still be subpar. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From tungwaiyip at yahoo.com Wed Feb 3 02:24:38 2010 From: tungwaiyip at yahoo.com (Tung Wai Yip) Date: Tue, 02 Feb 2010 17:24:38 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <20100203011538.GA11207@panix.com> References: <20100202221425.GA7800@panix.com> <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> <20100203011538.GA11207@panix.com> Message-ID: Sorry Crshonda. I don't mean to hijack your job ads :) As you can see this mailing list is very nerdy. Yes, the issue is about utilizing multicore CPU for computation intensive task. But it is also about writing clever one line remark to make fun of job ads ;-) Wai Yip > On Tue, Feb 02, 2010, Asher Langton wrote: >> On Feb 2, 2010, at 2:14 PM, Aahz wrote: >>> On Tue, Feb 02, 2010, Tung Wai Yip wrote: >>>> >>>> I think my experience tells me don't build multi-threaded >>>> applications in Python ;-) >>> >>> Why? I've had about equal trouble with multi-processing and >>> multi-threading. >> >> In short: multi-threading is really bad in Python 2.x when the threads >> are CPU-bound. > > Absolutely. Well, almost -- it's still really good for creating a > responsive GUI application. See my Fibonacci example in my threads > tutorial. > > Nevertheless, that's completely different from "don't use threads". > > PS: you can remove the "2.x" from your comment; even if the Unladen > Swallow people improve GIL handling, it's almost certain that CPU-bound > threads will still be subpar. From langton2 at llnl.gov Wed Feb 3 02:25:15 2010 From: langton2 at llnl.gov (Asher Langton) Date: Tue, 2 Feb 2010 17:25:15 -0800 Subject: [Baypiggies] Number of inodes on a volume? In-Reply-To: References: Message-ID: On Feb 2, 2010, at 5:13 PM, Glen Jarvis wrote: > I have a quick question to this brilliant group :) > > How do I, using pure python, get the number of inodes free on a volume > for a particular mount? > > Basically, is there an equivalent to df -i in Python? Is this > somewhere in stat and I can't find it? Take a look at os.statvfs(). -Asher From tnm800 at gmail.com Wed Feb 3 02:26:47 2010 From: tnm800 at gmail.com (Ted Nyman) Date: Tue, 2 Feb 2010 17:26:47 -0800 Subject: [Baypiggies] Number of inodes on a volume? In-Reply-To: References: Message-ID: <3fdfa9d61002021726m3d4b7659g79a71076d8042363@mail.gmail.com> I suppose there are a few ways; I'd say statvfs is probably the best way. If you want to try returning the free bytes, how about: import os import statvfs def get_freebytes(path): the_data = os.statvfs(path) return the_data[statvfs.F_BSIZE] * the_data[statvfs.F_BFREE] -Ted Nyman On Tue, Feb 2, 2010 at 5:13 PM, Glen Jarvis wrote: > I have a quick question to this brilliant group :) > > How do I, using pure python, get the number of inodes free on a volume > for a particular mount? > > Basically, is there an equivalent to df -i in Python? Is this > somewhere in stat and I can't find it? > > > Cheers, > > > Glen > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From glen at glenjarvis.com Wed Feb 3 02:28:35 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Tue, 2 Feb 2010 17:28:35 -0800 Subject: [Baypiggies] Number of inodes on a volume? In-Reply-To: References: Message-ID: Thanks Asher and Aaron... You're awesome! :) Glen On 2/2/10, Asher Langton wrote: > On Feb 2, 2010, at 5:13 PM, Glen Jarvis wrote: > > > I have a quick question to this brilliant group :) > > > > How do I, using pure python, get the number of inodes free on a volume > > for a particular mount? > > > > Basically, is there an equivalent to df -i in Python? Is this > > somewhere in stat and I can't find it? > > > > > Take a look at os.statvfs(). > > > -Asher > From aaron at merfinllc.com Wed Feb 3 02:24:35 2010 From: aaron at merfinllc.com (Aaron Straus) Date: Tue, 2 Feb 2010 17:24:35 -0800 Subject: [Baypiggies] Number of inodes on a volume? In-Reply-To: References: Message-ID: <99f0be831002021724m571bb73fk2e32926c1a504e41@mail.gmail.com> On Tue, Feb 2, 2010 at 5:13 PM, Glen Jarvis wrote: > How do I, using pure python, get the number of inodes free on a volume > for a particular mount? os.statvfs('.').f_ffree Cheers =a= -- =================== Aaron Straus aaron at merfinllc.com From jesse_gough at symantec.com Wed Feb 3 02:29:01 2010 From: jesse_gough at symantec.com (Jesse Gough) Date: Tue, 2 Feb 2010 17:29:01 -0800 Subject: [Baypiggies] Number of inodes on a volume? In-Reply-To: References: Message-ID: <20100203012900.GA23121@symantec.com> On Tue, 02 Feb 2010, Glen Jarvis wrote: > I have a quick question to this brilliant group :) > > How do I, using pure python, get the number of inodes free on a volume > for a particular mount? > > Basically, is there an equivalent to df -i in Python? Is this > somewhere in stat and I can't find it? os.statvfs() will do what you need. It's the "f_ffree" member. Pydoc isn't so helpful there, but you can give meaning to the statvfs_result attributes by looking at 'man statfs': struct statfs { long f_type; /* type of file system (see below) */ long f_bsize; /* optimal transfer block size */ long f_blocks; /* total data blocks in file system */ long f_bfree; /* free blocks in fs */ long f_bavail; /* free blocks avail to non-superuser */ long f_files; /* total file nodes in file system */ long f_ffree; /* free file nodes in fs */ fsid_t f_fsid; /* file system id */ long f_namelen; /* maximum length of filenames */ }; -Jesse From alecf at flett.org Wed Feb 3 04:26:34 2010 From: alecf at flett.org (Alec Flett) Date: Tue, 2 Feb 2010 19:26:34 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <78b3a9581002021620g5ced22c7u22fb2309f6602620@mail.gmail.com> References: <20100202221425.GA7800@panix.com> <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> <78b3a9581002021620g5ced22c7u22fb2309f6602620@mail.gmail.com> Message-ID: Wait, you guys (Aahz too?) have to see the full Dave Beazly talk! He really knows his stuff, too. Yes, it's basically single threaded on a single CPU machine... but in the talk he demonstrates how performance goes down dramatically as you add CPUs - basically the threads end up spending more time fighting for the GIL than actually use it, because multiple CPUs actually allow them to fight. On a single core machine, they play nicely and just take turns. http://blip.tv/file/2232410 I'd be really interested in seeing the Aahz's FIbbonaci example on a 8x machine with 7 threads working on Fibbonaci and one doing the UI :) I suspect it will actually slow to a crawl. Even Guido thought the talk was "amazing": http://twitter.com/gvanrossum/status/2348060772 Alec On Tue, Feb 2, 2010 at 4:20 PM, wesley chun wrote: > >> In short: multi-threading is really bad in Python 2.x when the threads > are CPU-bound. > > > some will even argue it *isn't* multithreading in this degenerative > case... it's just round-robin. > > -- wesley > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > "Python Web Development with Django", Addison Wesley, (c) 2009 > http://withdjango.com > > wesley.j.chun :: wescpy-at-gmail.com > python training and technical consulting > cyberweb.consulting : silicon valley, ca > http://cyberwebconsulting.com > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From glen at glenjarvis.com Wed Feb 3 05:30:47 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Tue, 2 Feb 2010 20:30:47 -0800 Subject: [Baypiggies] Python and MS Word docs; Argghhhhhhhhh...... Message-ID: No, that isn't pirate talk.. unless you want it to be... That is an interesting new problem that was put on my plate... at 8 pm... to be solved by morning.... argh matey... Unfortunately, many bioinformatics teams have a disconnect between computer science and biologists... For example, I, as a computer scientist, sometimes hear Charlie Brown's teacher (mwah wah wah mwah) and don't understand even what I'm supposed to do (and thus I should take more biology courses)... And, the biologists sometimes don't understand the benefit and limitations of technology and what they are asking for... One of the things that has been requested of us to take an MS word file that has been used to enter plain text. The file should be uploaded via webpage (done). The file should strip out all of the MS Word formatting so that we process only the text (And why not just upload a plain text file again? hmm? This is what is *really* wanted). In my introduction to Python a few years ago, I remember reading that there are python modules to read MS Word. Can these libraries be run on Linux/Unix, or is a .NET framework needed (we're a Linux only shop)? Most importantly, can this be done? Please say we can do something like this?on any platform because Python rocks: from dot_net import MSWordDoc word_file = open('my_example.doc', 'r') word_doc = MSWordDoc(word_file) word_file.close() text_only = word_doc.convert_to_text(encoding='ascii') Obviously I made up that syntax. If anyone ever finds this on the web looking for the same answer, *don't* use the the above code.. It's fake... Cheers, Glen P.S. Bonus if I can get an equivalent of the Unix "file" utility: > file sillywalk.doc sillywalk.doc: Microsoft Office Document From aahz at pythoncraft.com Wed Feb 3 05:41:35 2010 From: aahz at pythoncraft.com (Aahz) Date: Tue, 2 Feb 2010 20:41:35 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: References: <20100202221425.GA7800@panix.com> <26447B1A-6552-4632-B702-4A7526C140DA@llnl.gov> <78b3a9581002021620g5ced22c7u22fb2309f6602620@mail.gmail.com> Message-ID: <20100203044135.GA19005@panix.com> On Tue, Feb 02, 2010, Alec Flett wrote: > > I'd be really interested in seeing the Aahz's FIbbonaci example on a 8x > machine with 7 threads working on Fibbonaci and one doing the UI :) I > suspect it will actually slow to a crawl. Yes, it would, but I would hope nobody would try doing that in a production application. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From aahz at pythoncraft.com Wed Feb 3 05:43:02 2010 From: aahz at pythoncraft.com (Aahz) Date: Tue, 2 Feb 2010 20:43:02 -0800 Subject: [Baypiggies] Python and MS Word docs; Argghhhhhhhhh...... In-Reply-To: References: Message-ID: <20100203044302.GB19005@panix.com> On Tue, Feb 02, 2010, Glen Jarvis wrote: > > One of the things that has been requested of us to take an MS word > file that has been used to enter plain text. The file should be > uploaded via webpage (done). The file should strip out all of the MS > Word formatting so that we process only the text (And why not just > upload a plain text file again? hmm? This is what is *really* wanted). What I'd recommend is installing antiword and calling it from Python. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From jeff.enderwick at gmail.com Wed Feb 3 05:49:44 2010 From: jeff.enderwick at gmail.com (Jeff Enderwick) Date: Tue, 2 Feb 2010 20:49:44 -0800 Subject: [Baypiggies] Python and MS Word docs; Argghhhhhhhhh...... In-Reply-To: References: Message-ID: That sucks! In all seriousness, if you only have to support the later 'docx' format, those things are actually a zip file packed with XML. Inside you can find the text in an XML file (I think document.xml) and fish out the text. If you have to support the older formats, there is an Apache POI project (I have not played with it). When I was up against this, I had the docs saved "as a web page" and then consumed them with Beautiful Soup. I needed structure, style tags, etc, and 'Soup did well by me. On Tue, Feb 2, 2010 at 8:30 PM, Glen Jarvis wrote: > No, that isn't pirate talk.. unless you want it to be... > > That is an interesting new problem that was put on my plate... at 8 > pm... to be solved by morning.... argh matey... > > Unfortunately, many bioinformatics teams have a disconnect between > computer science and biologists... For example, I, as a computer > scientist, sometimes hear Charlie Brown's teacher (mwah wah wah mwah) > and don't understand even what I'm supposed to do (and thus I should > take more biology courses)... And, the biologists sometimes don't > understand the benefit and limitations of technology and what they are > asking for... > > One of the things that has been requested of us to take an MS word > file that has been used to enter plain text. The file should be > uploaded via webpage (done). The file should strip out all of the MS > Word formatting so that we process only the text (And why not just > upload a plain text file again? hmm? This is what is *really* wanted). > > In my introduction to Python a few years ago, I remember reading that > there are python modules to read MS Word. Can these libraries be run > on Linux/Unix, or is a .NET framework needed (we're a Linux only > shop)? > > Most importantly, can this be done? Please say we can do something > like this on any platform because Python rocks: > > from dot_net import MSWordDoc > > word_file = open('my_example.doc', 'r') > word_doc = MSWordDoc(word_file) > word_file.close() > > text_only = word_doc.convert_to_text(encoding='ascii') > > Obviously I made up that syntax. If anyone ever finds this on the web > looking for the same answer, *don't* use the the above code.. It's > fake... > > Cheers, > > > Glen > P.S. Bonus if I can get an equivalent of the Unix "file" utility: > > file sillywalk.doc > sillywalk.doc: Microsoft Office Document > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gallo.j at gmail.com Wed Feb 3 05:59:52 2010 From: gallo.j at gmail.com (joe gallo) Date: Tue, 2 Feb 2010 20:59:52 -0800 Subject: [Baypiggies] Python and MS Word docs; Argghhhhhhhhh...... In-Reply-To: References: Message-ID: <4b6381421002022059r2ebcf196r367bedf4d61f6f95@mail.gmail.com> > In my introduction to Python a few years ago, I remember reading that > there are python modules to read MS Word. Can these libraries be run > on Linux/Unix, or is a .NET framework needed (we're a Linux only > shop)? > http://stackoverflow.com/questions/125222/extracting-text-from-ms-word-files-in-python http://stackoverflow.com/questions/910730/python-ms-word -------------- next part -------------- An HTML attachment was scrubbed... URL: From p at ulmcnett.com Wed Feb 3 06:03:26 2010 From: p at ulmcnett.com (Paul McNett) Date: Tue, 02 Feb 2010 21:03:26 -0800 Subject: [Baypiggies] Python and MS Word docs; Argghhhhhhhhh...... In-Reply-To: References: Message-ID: <4B69039E.9060200@ulmcnett.com> On 2/2/10 8:30 PM, Glen Jarvis wrote: > One of the things that has been requested of us to take an MS word > file that has been used to enter plain text. The file should be > uploaded via webpage (done). The file should strip out all of the MS > Word formatting so that we process only the text (And why not just > upload a plain text file again? hmm? This is what is *really* wanted). Sheesh, they can't be bothered to *copy* and *paste* the text from their word doc to a web form? That sure sounds easier than navigating the filesystem to upload a file... Paul From jeremy.r.fishman at gmail.com Wed Feb 3 06:21:57 2010 From: jeremy.r.fishman at gmail.com (Jeremy Fishman) Date: Tue, 2 Feb 2010 21:21:57 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: References: Message-ID: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> My experience tells me something slightly different about Python: * for parallel processing, use multiple processes, ideally one per processor * for responsive I/O with an intuitive model, use threads * for high throughput I/O, use asynchronous I/O and/or an event framework Plenty of relatively high performance systems use one or a combination of the above. Clearly some use cases, especially in scientific computing, require tight parallel loops on local memory. I find it hard to believe numpy+boost can't go the distance. - Jeremy -- Jeremy Fishman Quantcast Corporation jeremy.r.fishman at gmail.com On Tue, Feb 2, 2010 at 1:28 PM, Tung Wai Yip wrote: > On Tue, 02 Feb 2010 12:17:57 -0800, Crshonda Williams < > crshonda at hotmail.com> wrote: > > >> Please send resumes to: crshonda at chompapps.com >> Crawler/Backend Engineer >> Chomp >> San Francisco, CA >> >> .... > > Experience >> - Strong Python skills. Experience with building multi-threaded >> applications in Python >> > > > I think my experience tells me don't build multi-threaded applications in > Python ;-) > > > Wai Yip > > > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From keith at dartworks.biz Wed Feb 3 06:35:14 2010 From: keith at dartworks.biz (Keith Dart) Date: Tue, 2 Feb 2010 21:35:14 -0800 Subject: [Baypiggies] Python and MS Word docs; Argghhhhhhhhh...... In-Reply-To: <4B69039E.9060200@ulmcnett.com> References: <4B69039E.9060200@ulmcnett.com> Message-ID: <20100202213514.385e8ed8@dartworks.biz> === On Tue, 02/02, Paul McNett wrote: === > Sheesh, they can't be bothered to *copy* and *paste* the text from > their word doc to a web form? That sure sounds easier than navigating > the filesystem to upload a file... === Even better would be a web application they can enter their data directly into, bypassing Word altogether. -- Keith Dart -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: 19017044 ===================================================================== From mvoorhie at yahoo.com Wed Feb 3 08:08:01 2010 From: mvoorhie at yahoo.com (Mark Voorhies) Date: Tue, 2 Feb 2010 23:08:01 -0800 Subject: [Baypiggies] Python and MS Word docs; Argghhhhhhhhh...... In-Reply-To: <20100203044302.GB19005@panix.com> References: <20100203044302.GB19005@panix.com> Message-ID: <201002022308.01794.mvoorhie@yahoo.com> On Tuesday 02 February 2010 08:43:02 pm Aahz wrote: > On Tue, Feb 02, 2010, Glen Jarvis wrote: > > One of the things that has been requested of us to take an MS word > > file that has been used to enter plain text. The file should be > > uploaded via webpage (done). The file should strip out all of the MS > > Word formatting so that we process only the text (And why not just > > upload a plain text file again? hmm? This is what is *really* wanted). > > What I'd recommend is installing antiword and calling it from Python. > catdoc is another command-line program in a similar vein. Both catdoc and antiword are apt-getable from the Debian and Ubuntu repositories. From glen at glenjarvis.com Wed Feb 3 08:32:10 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Tue, 2 Feb 2010 23:32:10 -0800 Subject: [Baypiggies] Python and MS Word docs; Argghhhhhhhhh...... In-Reply-To: <201002022308.01794.mvoorhie@yahoo.com> References: <20100203044302.GB19005@panix.com> <201002022308.01794.mvoorhie@yahoo.com> Message-ID: <49F5A943-9BE0-429F-8FA4-317038EDB56E@glenjarvis.com> Thank you everyone for such great suggestions. I actually found a human-based work around for this evening. After some discussion with the user, I finally realized why MS Word was so important and was able to give a work-around with notepad and TextEdit. I might still need to implement this word -> text pipeline, but not tonight. Thank you again for all of your suggestions... I'm headed home..... And it's not even the last BART :) (by 15 min)... Glen On Feb 2, 2010, at 11:08 PM, Mark Voorhies wrote: > On Tuesday 02 February 2010 08:43:02 pm Aahz wrote: >> On Tue, Feb 02, 2010, Glen Jarvis wrote: >>> One of the things that has been requested of us to take an MS word >>> file that has been used to enter plain text. The file should be >>> uploaded via webpage (done). The file should strip out all of the MS >>> Word formatting so that we process only the text (And why not just >>> upload a plain text file again? hmm? This is what is *really* >>> wanted). >> >> What I'd recommend is installing antiword and calling it from Python. >> > > catdoc is another command-line program in a similar vein. > > Both catdoc and antiword are apt-getable from the Debian and Ubuntu > repositories. > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies From jeff at drinktomi.com Wed Feb 3 11:10:21 2010 From: jeff at drinktomi.com (Jeff Younker) Date: Wed, 3 Feb 2010 02:10:21 -0800 Subject: [Baypiggies] Python and MS Word docs; Argghhhhhhhhh...... In-Reply-To: References: Message-ID: Python Win32 running on Windows is your friend. You can use COM objects just as if they were Python objects, and MS exposes pretty much all of the functionality of Excel, MS Word, etc. as COM objects. When it comes to manipulating and scripting end-user applications Microsoft has the UNIX world beat. Hands down, no competition. They've already solved application integration problems that the UNIX world hasn't even recognized as existing. Don't try to use OpenOffice. Don't try to use NeoOffice. Don't bother trying to use the open source solutions. Just learn how to extract the data directly from Word/Excel/etc. on the Windows side (with Python), and then feed that data over to the UNIX world. Working with application level integration on the Windows side is actually quite enjoyable and pleasurable. -jeff [ My experience indicates that only MS Office apps are actually capable of interpreting MS Office documents in a manner that is consistent with the user's reality. I've attempted to use OpenOffice and NeoOffice, and they were utter failures once I moved passed basic functionality.] From rich at noir.com Wed Feb 3 18:33:57 2010 From: rich at noir.com (K. Richard Pixley) Date: Wed, 03 Feb 2010 09:33:57 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> Message-ID: <4B69B385.2060808@noir.com> Jeremy Fishman wrote: > My experience tells me something slightly different about Python: > * for parallel processing, use multiple processes, ideally one per > processor > * for responsive I/O with an intuitive model, use threads > * for high throughput I/O, use asynchronous I/O and/or an event > framework Even apart from python, that's pretty much my experience across the board with other languages and other systems. Multiple heavy weight processes have the advantage that they can be spread across kernel instances which allows for hardware expansion, allocating varying degrees of hardware power to different processes, etc. I'll add that threads are generally less portable, much harder to debug, (debuggers never work quite right), much harder to test, (those silly nondeterminisms!), and that it's just as easy to architect an event framework around asynchronous I/O which is just as intuitive as a threaded model. (I'm doing this now for a python project). I'm pretty down on threads. I think of them as a nasty unprofessionalism akin to "goto" and symbolic links. I think it's possible to write reasonable code with them. I just haven't found the environment where that was easier or more straightforward than doing so with async I/O or multiple heavy weight processes. --rich From jeff.enderwick at gmail.com Wed Feb 3 19:36:57 2010 From: jeff.enderwick at gmail.com (Jeff Enderwick) Date: Wed, 3 Feb 2010 10:36:57 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <4B69B385.2060808@noir.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> Message-ID: I think you insult "goto" with your statements below :-). I too have had good experiences with event-based frameworks. You can avoid all that thread pool stuff, and your state ends up in normal data structures that can be easily dumped and displayed (versus some thread's stack frame). On Wed, Feb 3, 2010 at 9:33 AM, K. Richard Pixley wrote: > Jeremy Fishman wrote: > >> My experience tells me something slightly different about Python: >> * for parallel processing, use multiple processes, ideally one per >> processor >> * for responsive I/O with an intuitive model, use threads >> * for high throughput I/O, use asynchronous I/O and/or an event framework >> > Even apart from python, that's pretty much my experience across the board > with other languages and other systems. > > Multiple heavy weight processes have the advantage that they can be spread > across kernel instances which allows for hardware expansion, allocating > varying degrees of hardware power to different processes, etc. > > I'll add that threads are generally less portable, much harder to debug, > (debuggers never work quite right), much harder to test, (those silly > nondeterminisms!), and that it's just as easy to architect an event > framework around asynchronous I/O which is just as intuitive as a threaded > model. (I'm doing this now for a python project). > > I'm pretty down on threads. I think of them as a nasty unprofessionalism > akin to "goto" and symbolic links. I think it's possible to write > reasonable code with them. I just haven't found the environment where that > was easier or more straightforward than doing so with async I/O or multiple > heavy weight processes. > > --rich > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich at noir.com Thu Feb 4 03:53:04 2010 From: rich at noir.com (K. Richard Pixley) Date: Wed, 03 Feb 2010 18:53:04 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> Message-ID: <4B6A3690.3050507@noir.com> Jeff Enderwick wrote: > I think you insult "goto" with your statements below :-). I too have > had good experiences with event-based frameworks. You can avoid all > that thread pool stuff, and your state ends up in normal data > structures that can be easily dumped and displayed (versus some > thread's stack frame). It just occurred to me to ask... What experience do folks have with event based frameworks in python? I was thinking I was going to have to write my own, but I stumbled upon "circuits" today and then read through some of the "twisted" doc and was surprised to find that it closely mirrored my own thoughts. Are there others? Are any of the implementations definitive either positively or negatively? --rich From keith at dartworks.biz Thu Feb 4 04:17:03 2010 From: keith at dartworks.biz (Keith Dart) Date: Wed, 3 Feb 2010 19:17:03 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <4B6A3690.3050507@noir.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> Message-ID: <20100203191703.0dcfda61@dartworks.biz> === On Wed, 02/03, K. Richard Pixley wrote: === > What experience do folks have with event based frameworks in python? > > I was thinking I was going to have to write my own, but I stumbled > upon "circuits" today and then read through some of the "twisted" doc > and was surprised to find that it closely mirrored my own thoughts. > > Are there others? Are any of the implementations definitive either > positively or negatively? === Twisted seems pretty good, and is fairly popular for that sort of thing. Some time ago I wrote one, which is part of Pycopia. I still use that one myself. I think I'm the only one... Although the powerdroid project also uses it for a test and measurement application (which I also wrote). If you're interested: http://code.google.com/p/pycopia/ core async module: http://code.google.com/p/pycopia/source/browse/trunk/core/pycopia/asyncio.py Which is mostly a simplifying wrapper around the stock "poll" module. It defines an interface that the rest of the framework follows. In addition to that there is support for timers (the itimer module) that work properly in an async environment and proper handling of EAGAIN throughout the framework (requires its own socket module, for example). One example of its use is the SNMP trap receiver: http://code.google.com/p/pycopia/source/browse/trunk/SNMP/pycopia/SNMP/traps.py For powerdroid: http://code.google.com/p/powerdroid/ An async example that also uses the realtime clock to pace measurements accurately: http://code.google.com/p/powerdroid/source/browse/trunk/src/droid/measure/sequencer.py Then there is (or was) Stackless Python... -- Keith Dart -- -- -------------------- Keith Dart ======================= From jeremy.r.fishman at gmail.com Thu Feb 4 05:54:13 2010 From: jeremy.r.fishman at gmail.com (Jeremy Fishman) Date: Wed, 3 Feb 2010 20:54:13 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <20100203191703.0dcfda61@dartworks.biz> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> <20100203191703.0dcfda61@dartworks.biz> Message-ID: <1f7713e31002032054g1628c179qd0e7d37084d5415d@mail.gmail.com> I'm a big fan of Twisted, though I don't know much what's going on there right now (they're overdue on their 2010 release - http://twistedmatrix.com/trac/milestone/Twisted-10.0). Most of the team works for Canonical now I believe. On the other hand, I find greenlets (http://pypi.python.org/pypi/greenlet) from Stackless pretty darn cool, although part of me says that they're just 'goto's for Python, which Rich may want to avoid. And on that note, I too think rich was a bit harsh on 'goto's. I definitively avoid them in general code, but occasionally they can serve the opposite purpose than that which their reputation predicts. A goto or two can clean up very specifically confusing code, such as when a several cases in a switch statement share different chunks of exit logic, but can't simple fall through. Sure, setting some flags and 'break'ing, or passing state to a helper function, would achieve the same thing, but it can clearly be cleaner and more understandable to 'goto label_foo' in the switch and then have some nice unindented cleanup labels before the function close. - Jeremy -- Jeremy R Fishman Quantcast Corporation jeremy.r.fishman at gmail.com On Wed, Feb 3, 2010 at 7:17 PM, Keith Dart wrote: > === On Wed, 02/03, K. Richard Pixley wrote: === > > What experience do folks have with event based frameworks in python? > > > > I was thinking I was going to have to write my own, but I stumbled > > upon "circuits" today and then read through some of the "twisted" doc > > and was surprised to find that it closely mirrored my own thoughts. > > > > Are there others? Are any of the implementations definitive either > > positively or negatively? > === > > Twisted seems pretty good, and is fairly popular for that sort of thing. > > Some time ago I wrote one, which is part of Pycopia. I still use that > one myself. I think I'm the only one... Although the powerdroid project > also uses it for a test and measurement application (which I also > wrote). > > If you're interested: > > http://code.google.com/p/pycopia/ > > core async module: > > http://code.google.com/p/pycopia/source/browse/trunk/core/pycopia/asyncio.py > > Which is mostly a simplifying wrapper around the stock "poll" module. > It defines an interface that the rest of the framework follows. In > addition to that there is support for timers (the itimer module) that > work properly in an async environment and proper handling of EAGAIN > throughout the framework (requires its own socket module, for example). > > One example of its use is the SNMP trap receiver: > > > http://code.google.com/p/pycopia/source/browse/trunk/SNMP/pycopia/SNMP/traps.py > > > For powerdroid: > http://code.google.com/p/powerdroid/ > > An async example that also uses the realtime clock to pace > measurements accurately: > > > http://code.google.com/p/powerdroid/source/browse/trunk/src/droid/measure/sequencer.py > > > Then there is (or was) Stackless Python... > > -- Keith Dart > > -- > -- -------------------- > Keith Dart > > ======================= > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From keith at dartworks.biz Thu Feb 4 06:12:17 2010 From: keith at dartworks.biz (Keith Dart) Date: Wed, 3 Feb 2010 21:12:17 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <1f7713e31002032054g1628c179qd0e7d37084d5415d@mail.gmail.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> <20100203191703.0dcfda61@dartworks.biz> <1f7713e31002032054g1628c179qd0e7d37084d5415d@mail.gmail.com> Message-ID: <20100203211217.19a2ec01@dartworks.biz> === On Wed, 02/03, Jeremy Fishman wrote: === > And on that note, I too think rich was a bit harsh on 'goto's. ?I > definitively avoid them in general code, but occasionally they can > serve the opposite purpose than that which their reputation predicts. > ?A goto or two can clean up very specifically confusing code, such as > when a several cases in a switch statement share different chunks of > exit logic, but can't simple fall through. ?Sure, setting some flags > and 'break'ing, or passing state to a helper function, would achieve > the same thing, but it can clearly be cleaner and more understandable > to 'goto label_foo' in the switch and then have some nice unindented > cleanup labels before the function close. === But in a language/runtime that has exceptions, like Python, you don't really need them. Just raise an exception where you would have the "goto", and catch it where you would have the label. I think that is even more readable than the goto. -- Keith Dart -- -- -------------------- Keith Dart ======================= From janssen at parc.com Thu Feb 4 18:11:10 2010 From: janssen at parc.com (Bill Janssen) Date: Thu, 4 Feb 2010 09:11:10 PST Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <4B6A3690.3050507@noir.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> Message-ID: <53611.1265303470@parc.com> K. Richard Pixley wrote: > What experience do folks have with event based frameworks in python? I'm perhaps one of the last users of Medusa, which is based on the asyncore/asynchat framework in the Python stdlib (and by the same author). It's what powers UpLib RPC and Web and IMAP interfaces (http://uplib.parc.com). I've been pretty happy with it overall. Bill From alecf at flett.org Thu Feb 4 18:25:43 2010 From: alecf at flett.org (Alec Flett) Date: Thu, 4 Feb 2010 09:25:43 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <4B6A3690.3050507@noir.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> Message-ID: > > > > What experience do folks have with event based frameworks in python? > > On the HTTP server side, Twisted is by far the most pervasive, but Tornado, which came out last year, has had a lot of positive responses, and I've heard of more and more people using it. In general the trade off with these is you have is that you have to think more about parallelism even during a single http request because your control flow changes dramatically - learn about Deferreds to learn more. The bonus is that you get that parallelism automatically, very cheaply, once you buy into the framework... but the drawback is that that because your control flow is no longer stack-based, WSGI generally goes out the window... so it's virtually impossible to reuse all those great WSGI middleware packages out there. Stackless has an excellent solution for this which is to make it look like your control flow follows the stack, and thus you can actually use WSGI middleware. Why is the "automatic parallelism" I mentioned above important? At least where I'm working, a single incoming HTTP request is fulfilled by the appserver making several (2-100) internal HTTP requests to internal services. If *those* requests can be made in parallel, then you can really get very high throughput. Our solution was actually to run Twisted inside of Pylons, but that's a longer discussion... Alec > I was thinking I was going to have to write my own, but I stumbled upon > "circuits" today and then read through some of the "twisted" doc and was > surprised to find that it closely mirrored my own thoughts. > > Are there others? Are any of the implementations definitive either > positively or negatively? > > --rich > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From spmcinerney at hotmail.com Thu Feb 4 18:35:42 2010 From: spmcinerney at hotmail.com (Stephen McInerney) Date: Thu, 4 Feb 2010 09:35:42 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: References: , , <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com>, <4B69B385.2060808@noir.com>, , <4B6A3690.3050507@noir.com>, Message-ID: Guys, Since this is a discussion of broad interest, why not have a night on head-to-head comparison of event-based frameworks and threading? (Jim is our next free date March?) Should I ask for a +1 if anyone agrees? Stephen Date: Thu, 4 Feb 2010 09:25:43 -0800 From: alecf at flett.org To: rich at noir.com CC: baypiggies at python.org Subject: Re: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA What experience do folks have with event based frameworks in python? On the HTTP server side, Twisted is by far the most pervasive, but Tornado, which came out last year, has had a lot of positive responses, and I've heard of more and more people using it. In general the trade off with these is you have is that you have to think more about parallelism even during a single http request because your control flow changes dramatically - learn about Deferreds to learn more. The bonus is that you get that parallelism automatically, very cheaply, once you buy into the framework... but the drawback is that that because your control flow is no longer stack-based, WSGI generally goes out the window... so it's virtually impossible to reuse all those great WSGI middleware packages out there. Stackless has an excellent solution for this which is to make it look like your control flow follows the stack, and thus you can actually use WSGI middleware. Why is the "automatic parallelism" I mentioned above important? At least where I'm working, a single incoming HTTP request is fulfilled by the appserver making several (2-100) internal HTTP requests to internal services. If *those* requests can be made in parallel, then you can really get very high throughput. Our solution was actually to run Twisted inside of Pylons, but that's a longer discussion... Alec I was thinking I was going to have to write my own, but I stumbled upon "circuits" today and then read through some of the "twisted" doc and was surprised to find that it closely mirrored my own thoughts. Are there others? Are any of the implementations definitive either positively or negatively? --rich _______________________________________________ Baypiggies mailing list Baypiggies at python.org To change your subscription options or unsubscribe: http://mail.python.org/mailman/listinfo/baypiggies _________________________________________________________________ Hotmail: Trusted email with powerful SPAM protection. http://clk.atdmt.com/GBL/go/201469227/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jjkunce at gmail.com Thu Feb 4 18:50:29 2010 From: jjkunce at gmail.com (Jeff Kunce) Date: Thu, 4 Feb 2010 09:50:29 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <53611.1265303470@parc.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> <53611.1265303470@parc.com> Message-ID: <833f4cd71002040950q7224cc10gdc9578df7047c178@mail.gmail.com> On Thu, Feb 4, 2010 at 9:11 AM, Bill Janssen wrote: > I'm perhaps one of the last users of Medusa, which is based on the Hardly :) How about anyone running the built-in webserver in Plone/Zope! Actually, I'm wondering about the scope of the original question about an event-based framework: - Just wanting objects to communicate with events? Something simple like circuits is good. - async IO? asyncore, medusa et al are tried and true - A comprehensive framework with everything included (even peanut butter and jelly)? Twisted The scope really matters. If you just want to send simple events between objects in your app, you could write your own circuits-like framework before you could even figure out how to read the documentation for Twisted. -- Jeff > asyncore/asynchat framework in the Python stdlib (and by the same > author). ?It's what powers UpLib RPC and Web and IMAP interfaces > (http://uplib.parc.com). ?I've been pretty happy with it overall. From rich at noir.com Thu Feb 4 18:56:02 2010 From: rich at noir.com (K. Richard Pixley) Date: Thu, 04 Feb 2010 09:56:02 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <20100203211217.19a2ec01@dartworks.biz> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> <20100203191703.0dcfda61@dartworks.biz> <1f7713e31002032054g1628c179qd0e7d37084d5415d@mail.gmail.com> <20100203211217.19a2ec01@dartworks.biz> Message-ID: <4B6B0A32.5030705@noir.com> Keith Dart wrote: > But in a language/runtime that has exceptions, like Python, you don't > really need them. Just raise an exception where you would have the > "goto", and catch it where you would have the label. I think that is > even more readable than the goto. Precisely. I've also found that in any complex logic where a goto might be called for, the complexity itself was effectively a bug both in code transparency as well as in performance. Rewriting such code to avoid the use of goto has, in every single case for me, yielded not only more transparent code, but also a smaller code foot print and more efficiently running code. Flags are a similar issue. If you need to create, set, and test flags solely in order to maintain your control flow, then that section of code almost certainly needs to be redesigned from a higher level, imo. I'll admit there are functional exceptions. In assembler. And occasionally those same exceptions can be coded in C. Things like processor bootup code, initial stack construction, the framing code around unix processes, process load code, unexec, certain trampolines (including those dealing with calls between processor architectures as with arm/thumb, implementations of shared libraries, kernel/user space transitions, some kinds of coprocessor support, varying or changing call semantics, etc), implementations of things like setjmp/longjmp, exceptions, continuations, jit emulations like qemu, etc. But aside from academic curiosity or thrill seeking, I don't see much point or need for doing so in python. I once had a friend who thought it would be cool to write a C compiler in shell. To me that's sort of like tearing a Ferrari apart solely with a pair of pliers. While it might be possible, and a challenge, it seems to me that one could find similar challenging possibilities which /also/ result in useful end products. I suppose one could always just declare it to be "art". --rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at zachary.com Thu Feb 4 19:11:28 2010 From: david at zachary.com (David Creemer) Date: Thu, 4 Feb 2010 10:11:28 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <833f4cd71002040950q7224cc10gdc9578df7047c178@mail.gmail.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> <53611.1265303470@parc.com> <833f4cd71002040950q7224cc10gdc9578df7047c178@mail.gmail.com> Message-ID: <6B736018-2C11-46F8-93E7-83199C473EDD@zachary.com> Supervisor, which I can recommend, also uses it. See supervisord.org -- David On Feb 4, 2010, at 9:50 AM, Jeff Kunce wrote: > On Thu, Feb 4, 2010 at 9:11 AM, Bill Janssen wrote: >> I'm perhaps one of the last users of Medusa, which is based on the > From rich at noir.com Thu Feb 4 19:28:24 2010 From: rich at noir.com (K. Richard Pixley) Date: Thu, 04 Feb 2010 10:28:24 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <833f4cd71002040950q7224cc10gdc9578df7047c178@mail.gmail.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> <53611.1265303470@parc.com> <833f4cd71002040950q7224cc10gdc9578df7047c178@mail.gmail.com> Message-ID: <4B6B11C8.90105@noir.com> Jeff Kunce wrote: > Actually, I'm wondering about the scope of the original question about > an event-based framework: > - Just wanting objects to communicate with events? Something simple > like circuits is good. > - async IO? asyncore, medusa et al are tried and true > - A comprehensive framework with everything included (even peanut > butter and jelly)? Twisted > > The scope really matters. If you just want to send simple events > between objects in your app, you could write your own circuits-like > framework before you could even figure out how to read the > documentation for Twisted. I'm building a moderately complex automated builder based primarily on incremental builds. Multiple servers, each parallel in their own right, building multiple code branches against multiple target machines. So I've got three or four distinct levels of parallelism going on in the builds already. And there are several types of checking going on, including precommit speculative testing, post commit "continuous builds", a form of partial building used for dependency checking, and a sort of pre-pre-commit checking, (essentially branched recurrences of the pre-commit checker). The builder monitors a source code repository building when necessary as well as taking speculative build requests from users and building those. Most of this parallelism doesn't happen in python. The python pieces of the builder coordinate the various builders, partition and aggregate very high level parallelism, manage working directories, and alerting mechanisms. My plan is a distributed, dynamic, asynchronous, fault tolerant architecture based on virtual synchrony, (http://en.wikipedia.org/wiki/Virtual_synchrony - probably the spread toolkit http://spread.org). There's also a set of status and input web pages, (javascript based on ext-js grid widget), and my plan is to produce their data using pylons, perhaps tg2, which pulls data out of the virtual synchrony "cloud", probably via spread, but potentially via traditional socket calls, maybe even jsonrpc. So the "daemon"/cloud part needs to be capable of managing distributed, shared state, including the work queue, through spread, as well as managing a few subtasks like builds. I'm not big on threads, and they don't really help much in the virtual synchrony paradigm anyway. It's not a /huge/ project, but I'm expecting it to take me at least a few months to implement. --rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich at noir.com Thu Feb 4 19:31:25 2010 From: rich at noir.com (K. Richard Pixley) Date: Thu, 04 Feb 2010 10:31:25 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: References: , , <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com>, <4B69B385.2060808@noir.com>, , <4B6A3690.3050507@noir.com>, Message-ID: <4B6B127D.5030209@noir.com> Stephen McInerney wrote: > Should I ask for a +1 if anyone agrees? I'm not opposed, but I have a strong preference for the online discussion format we're having now. I'd be happy to summarize and present on the topic later, or to participate in a group that did, if there's interest, so long as that doesn't impact current discussions. --rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich at noir.com Thu Feb 4 19:40:50 2010 From: rich at noir.com (K. Richard Pixley) Date: Thu, 04 Feb 2010 10:40:50 -0800 Subject: [Baypiggies] virtual synchrony In-Reply-To: <4B6B127D.5030209@noir.com> References: , , <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com>, <4B69B385.2060808@noir.com>, , <4B6A3690.3050507@noir.com>, <4B6B127D.5030209@noir.com> Message-ID: <4B6B14B2.5030801@noir.com> I'd also be willing to present on virtual synchrony. It's an idea from the 80's which largely predates threads and has never caught on for reasons I've never understood. In a nutshell, virtual synchrony is a system based on the idea of "reliable broadcast", except that the definition of "reliable" has been perverted. Instead of declaring that all receivers receive a broadcast "at the same time", (which isn't really possible in a 4d time space continuum except for very coarse grained definitions of "same time"), virtual synchrony guarantees that all receivers receive all messages in precisely the same order, (or they cease to participate in the virtual synchrony). As a base mechanism for coordination, virtual synchrony can be used to build simple operating system concepts like mutual exclusion, shared state, reliable publish/subscribe mechanisms, etc. The usual application is for distributed, asynchronous, parallel, fault tolerant, network based services. --rich From jjkunce at gmail.com Thu Feb 4 20:24:05 2010 From: jjkunce at gmail.com (Jeff Kunce) Date: Thu, 4 Feb 2010 11:24:05 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <4B6B11C8.90105@noir.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> <53611.1265303470@parc.com> <833f4cd71002040950q7224cc10gdc9578df7047c178@mail.gmail.com> <4B6B11C8.90105@noir.com> Message-ID: <833f4cd71002041124h19694913h28f81dd7ae93b6f8@mail.gmail.com> OK. It's not a simple messaging problem, and it's not primarily an async IO problem. If it were me, I'd spend a few hours researching Twisted. Read some of the online reviews/tutorials about it, then dig into the real thing. It's a pretty impressive framework. The most common complaint is that it has its own (somewhat "twisted") culture/terminology that makes it hard to learn. Spread is cool. I've tried several times to find an excuse to use it :) -- Jeff On Thu, Feb 4, 2010 at 10:28 AM, K. Richard Pixley wrote: > Jeff Kunce wrote: > > Actually, I'm wondering about the scope of the original question about > an event-based framework: > - Just wanting objects to communicate with events? Something simple > like circuits is good. > - async IO? asyncore, medusa et al are tried and true > - A comprehensive framework with everything included (even peanut > butter and jelly)? Twisted > > The scope really matters. If you just want to send simple events > between objects in your app, you could write your own circuits-like > framework before you could even figure out how to read the > documentation for Twisted. > > I'm building a moderately complex automated builder based primarily on > incremental builds.? Multiple servers, each parallel in their own right, > building multiple code branches against multiple target machines.? So I've > got three or four distinct levels of parallelism going on in the builds > already.? And there are several types of checking going on, including > precommit speculative testing, post commit "continuous builds", a form of > partial building used for dependency checking, and a sort of pre-pre-commit > checking, (essentially branched recurrences of the pre-commit checker).? The > builder monitors a source code repository building when necessary as well as > taking speculative build requests from users and building those. > > Most of this parallelism doesn't happen in python.? The python pieces of the > builder coordinate the various builders, partition and aggregate very high > level parallelism, manage working directories, and alerting mechanisms. > > My plan is a distributed, dynamic, asynchronous, fault tolerant architecture > based on virtual synchrony, (http://en.wikipedia.org/wiki/Virtual_synchrony > - probably the spread toolkit http://spread.org). > > There's also a set of status and input web pages, (javascript based on > ext-js grid widget), and my plan is to produce their data using pylons, > perhaps tg2, which pulls data out of the virtual synchrony "cloud", probably > via spread, but potentially via traditional socket calls, maybe even > jsonrpc. > > So the "daemon"/cloud part needs to be capable of managing distributed, > shared state, including the work queue, through spread, as well as managing > a few subtasks like builds.? I'm not big on threads, and they don't really > help much in the virtual synchrony paradigm anyway. > > It's not a huge project, but I'm expecting it to take me at least a few > months to implement. > > --rich > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From aahz at pythoncraft.com Thu Feb 4 23:27:06 2010 From: aahz at pythoncraft.com (Aahz) Date: Thu, 4 Feb 2010 14:27:06 -0800 Subject: [Baypiggies] asynch design In-Reply-To: <4B69B385.2060808@noir.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> Message-ID: <20100204222706.GA7542@panix.com> On Wed, Feb 03, 2010, K. Richard Pixley wrote: > > I'll add that threads are generally less portable, much harder to debug, > (debuggers never work quite right), much harder to test, (those silly > nondeterminisms!), and that it's just as easy to architect an event > framework around asynchronous I/O which is just as intuitive as a > threaded model. (I'm doing this now for a python project). Beware of trying to define what is "intuitive" for anyone other than yourself. I've tried to wrap my head around event frameworks and I find it way more difficult than threads, particularly if you design threads around a message-passing architecture. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From damonmc at gmail.com Fri Feb 5 02:07:39 2010 From: damonmc at gmail.com (Damon McCormick) Date: Thu, 4 Feb 2010 17:07:39 -0800 Subject: [Baypiggies] Web Crawler/Backend Engineer - San Francisco, CA In-Reply-To: <833f4cd71002041124h19694913h28f81dd7ae93b6f8@mail.gmail.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> <53611.1265303470@parc.com> <833f4cd71002040950q7224cc10gdc9578df7047c178@mail.gmail.com> <4B6B11C8.90105@noir.com> <833f4cd71002041124h19694913h28f81dd7ae93b6f8@mail.gmail.com> Message-ID: <3cce76f61002041707q40db9abbmf034cbf43e465999@mail.gmail.com> I don't agree with all the presenter's conclusions, but this videomentions a bunch of asynchronous/event-based frameworks: Cogen, Multitask, Chiral, Eventlet, Weightless, Fibra, Stackless Python, Twisted, Concurrence, Circuits.Web, Greenlet, Candygram, Papyros, Parley. (I copied this list from 34:10) -Damon On Thu, Feb 4, 2010 at 11:24 AM, Jeff Kunce wrote: > OK. It's not a simple messaging problem, and it's not primarily an > async IO problem. If it were me, I'd spend a few hours researching > Twisted. Read some of the online reviews/tutorials about it, then dig > into the real thing. It's a pretty impressive framework. The most > common complaint is that it has its own (somewhat "twisted") > culture/terminology that makes it hard to learn. > > Spread is cool. I've tried several times to find an excuse to use it :) > > -- Jeff > > On Thu, Feb 4, 2010 at 10:28 AM, K. Richard Pixley wrote: > > Jeff Kunce wrote: > > > > Actually, I'm wondering about the scope of the original question about > > an event-based framework: > > - Just wanting objects to communicate with events? Something simple > > like circuits is good. > > - async IO? asyncore, medusa et al are tried and true > > - A comprehensive framework with everything included (even peanut > > butter and jelly)? Twisted > > > > The scope really matters. If you just want to send simple events > > between objects in your app, you could write your own circuits-like > > framework before you could even figure out how to read the > > documentation for Twisted. > > > > I'm building a moderately complex automated builder based primarily on > > incremental builds. Multiple servers, each parallel in their own right, > > building multiple code branches against multiple target machines. So > I've > > got three or four distinct levels of parallelism going on in the builds > > already. And there are several types of checking going on, including > > precommit speculative testing, post commit "continuous builds", a form of > > partial building used for dependency checking, and a sort of > pre-pre-commit > > checking, (essentially branched recurrences of the pre-commit checker). > The > > builder monitors a source code repository building when necessary as well > as > > taking speculative build requests from users and building those. > > > > Most of this parallelism doesn't happen in python. The python pieces of > the > > builder coordinate the various builders, partition and aggregate very > high > > level parallelism, manage working directories, and alerting mechanisms. > > > > My plan is a distributed, dynamic, asynchronous, fault tolerant > architecture > > based on virtual synchrony, ( > http://en.wikipedia.org/wiki/Virtual_synchrony > > - probably the spread toolkit http://spread.org). > > > > There's also a set of status and input web pages, (javascript based on > > ext-js grid widget), and my plan is to produce their data using pylons, > > perhaps tg2, which pulls data out of the virtual synchrony "cloud", > probably > > via spread, but potentially via traditional socket calls, maybe even > > jsonrpc. > > > > So the "daemon"/cloud part needs to be capable of managing distributed, > > shared state, including the work queue, through spread, as well as > managing > > a few subtasks like builds. I'm not big on threads, and they don't > really > > help much in the virtual synchrony paradigm anyway. > > > > It's not a huge project, but I'm expecting it to take me at least a few > > months to implement. > > > > --rich > > > > _______________________________________________ > > Baypiggies mailing list > > Baypiggies at python.org > > To change your subscription options or unsubscribe: > > http://mail.python.org/mailman/listinfo/baypiggies > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From donnamsnow at gmail.com Sat Feb 6 19:53:28 2010 From: donnamsnow at gmail.com (Donna Snow) Date: Sat, 6 Feb 2010 10:53:28 -0800 Subject: [Baypiggies] teach a django class? Message-ID: I have 7 people interested in learning Django - they signed up for my class but my instructors had to back out because there weren't enough people to make it worthwhile (at least for them) I can pay someone to teach based on the following syllabus - http://www.c2etraining.com/django_syllabus.doc If you are interested please contact me offlist. We will probably schedule the class for mid-march. Thank you! Donna 'SnowWrite' Snow -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeff.enderwick at gmail.com Sun Feb 7 09:08:03 2010 From: jeff.enderwick at gmail.com (Jeff Enderwick) Date: Sun, 7 Feb 2010 00:08:03 -0800 Subject: [Baypiggies] asynch design In-Reply-To: <20100204222706.GA7542@panix.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <20100204222706.GA7542@panix.com> Message-ID: I will say it took some time for the event-based mentality to seep into my brain. My background was more unix-y, and then I ended up at a router company where the code was almost exclusively under the control of event frameworks, with a run-to-completion model. Eventually, I was able to think in the event-based way, and I have benefitted. For me, threads/processes were more natural at first because I am human. At home, I might pick up a box, carry it over there, wait for my wife to open the door, then put the box down. I naturally think in terms of serial actions performed by my body, and this thought process carries over when I am thinking about what a program should do. But, IMO, it is not the best model for most programming. In your program you have state and events. When you have a programming architecture that specified "there are N threads that do " or "this thread does ", you are creating constrictions that aren't a natural/necessary part of the problem that the program is solving. $.02 On Thu, Feb 4, 2010 at 2:27 PM, Aahz wrote: > On Wed, Feb 03, 2010, K. Richard Pixley wrote: > > > > I'll add that threads are generally less portable, much harder to debug, > > (debuggers never work quite right), much harder to test, (those silly > > nondeterminisms!), and that it's just as easy to architect an event > > framework around asynchronous I/O which is just as intuitive as a > > threaded model. (I'm doing this now for a python project). > > Beware of trying to define what is "intuitive" for anyone other than > yourself. I've tried to wrap my head around event frameworks and I find > it way more difficult than threads, particularly if you design threads > around a message-passing architecture. > -- > Aahz (aahz at pythoncraft.com) <*> > http://www.pythoncraft.com/ > > import antigravity > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich at noir.com Mon Feb 8 18:04:54 2010 From: rich at noir.com (K. Richard Pixley) Date: Mon, 08 Feb 2010 09:04:54 -0800 Subject: [Baypiggies] asynch design In-Reply-To: References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <20100204222706.GA7542@panix.com> Message-ID: <4B704436.9000509@noir.com> My initial experience with either paradigm was coop, (ie, no preemptive scheduling). The entire original macos was cooperative event based. Most windows/icon/mice interfaces, (including X11), were originally event based. And the ISIS toolkit, (an early version of what is now called "virtual synchrony"), was a messaging system without a "read" call. Instead, incoming messages spawned a "thread", (really implemented as a coop event based tasking system). Also, my first asynchronous parallel system was back in the 80's with Sequent Balances and later Symmetries combined with Sun 3/50's. There were no widely available threading systems then although the coop threading systems were beginning to show up in various places. I designed and built an entire multiple machine, asynchronous ticker plant using heavy weight unix processes, sysv unix IPC, sockets, and hardware atomic locks. (And later replaced it with an ISIS based system). Early implementations of threads were cooperative and many were implemented using event based systems. That is, threads which blocked blocked the entire heavy weight process. At that point, the distinction between threads and an event based system was largely moot. If you wanted an asynchronous system, then you had to think about blocking and potential task switching with each and every call either way. Granted, modern threading systems are preemptive and much more heavy weight than the earlier systems. Threads have lost much of their efficiency in favor of a simplified user paradigm, (aside from debugging), which now virtually mimics the original unix process paradigm, (complete with shared memory, forks, multiple asynchronous processes, atomic hardware locking, etc). Personally, I'd prefer to use the unix facilities at that point because they are more efficient and easier to debug. From that perspective, threads offer very little in the way of functionality or efficiency over prior, well known alternatives. I agree that if you haven't used any of these paradigms, then threads might be a simpler paradigm to learn initially, if you assume that your threading system is part of your operating system or you assume that all threading systems are preemptive and therefor comparable. Event based systems force the developer to confront the task switching mechanism and to sort through starvation scenarios. If you're going to sort through those scenarios anyway, or if you're going to need to examine your switching mechanism closely, then I argue that event based systems are simpler to understand in total than thread based. So I suppose I agree that if you only need to understand a portion of the problem space, then threads are probably easier, (so long as you don't need to debug anything). --rich From trystanleftwich at gmail.com Mon Feb 8 18:16:54 2010 From: trystanleftwich at gmail.com (Trystan Leftwich) Date: Mon, 8 Feb 2010 09:16:54 -0800 Subject: [Baypiggies] Software Engineering positions at Crowd Science Message-ID: Hi everyone, We?re a 2 year old startup located in Mountain View, looking for both a Junior and Senior Engineer to join our team. Our products are built on Django with a fair amount of pure Python in the mix, along with the ubiquitous HTML, CSS and Javascript for the UI. More information can be found at http://www.crowdscience.com/jobs If your interested, please email your resume or any questions you have to engineering-jobs at crowdscience.com Regards, Trystan -------------- next part -------------- An HTML attachment was scrubbed... URL: From slacy at slacy.com Mon Feb 8 20:14:49 2010 From: slacy at slacy.com (Stephen Lacy) Date: Mon, 8 Feb 2010 11:14:49 -0800 Subject: [Baypiggies] asynch design In-Reply-To: <4B704436.9000509@noir.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <20100204222706.GA7542@panix.com> <4B704436.9000509@noir.com> Message-ID: <9b97850f1002081114j3103f67el89c29457706f41c6@mail.gmail.com> Here's a great (and recent) comparison of a whole bunch of asynchronous web servers in Python: http://nichol.as/asynchronous-servers-in-python (via http://firendfeed.com/amitp) Steve On Mon, Feb 8, 2010 at 9:04 AM, K. Richard Pixley wrote: > My initial experience with either paradigm was coop, (ie, no preemptive > scheduling). The entire original macos was cooperative event based. Most > windows/icon/mice interfaces, (including X11), were originally event based. > And the ISIS toolkit, (an early version of what is now called "virtual > synchrony"), was a messaging system without a "read" call. Instead, > incoming messages spawned a "thread", (really implemented as a coop event > based tasking system). > > Also, my first asynchronous parallel system was back in the 80's with > Sequent Balances and later Symmetries combined with Sun 3/50's. There were > no widely available threading systems then although the coop threading > systems were beginning to show up in various places. I designed and built > an entire multiple machine, asynchronous ticker plant using heavy weight > unix processes, sysv unix IPC, sockets, and hardware atomic locks. (And > later replaced it with an ISIS based system). > > Early implementations of threads were cooperative and many were implemented > using event based systems. That is, threads which blocked blocked the > entire heavy weight process. At that point, the distinction between threads > and an event based system was largely moot. If you wanted an asynchronous > system, then you had to think about blocking and potential task switching > with each and every call either way. > > Granted, modern threading systems are preemptive and much more heavy weight > than the earlier systems. Threads have lost much of their efficiency in > favor of a simplified user paradigm, (aside from debugging), which now > virtually mimics the original unix process paradigm, (complete with shared > memory, forks, multiple asynchronous processes, atomic hardware locking, > etc). Personally, I'd prefer to use the unix facilities at that point > because they are more efficient and easier to debug. From that perspective, > threads offer very little in the way of functionality or efficiency over > prior, well known alternatives. > > I agree that if you haven't used any of these paradigms, then threads might > be a simpler paradigm to learn initially, if you assume that your threading > system is part of your operating system or you assume that all threading > systems are preemptive and therefor comparable. Event based systems force > the developer to confront the task switching mechanism and to sort through > starvation scenarios. If you're going to sort through those scenarios > anyway, or if you're going to need to examine your switching mechanism > closely, then I argue that event based systems are simpler to understand in > total than thread based. > > So I suppose I agree that if you only need to understand a portion of the > problem space, then threads are probably easier, (so long as you don't need > to debug anything). > > --rich > > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From spmcinerney at hotmail.com Mon Feb 8 23:40:25 2010 From: spmcinerney at hotmail.com (Stephen McInerney) Date: Mon, 8 Feb 2010 14:40:25 -0800 Subject: [Baypiggies] Who's going to PyCon? Message-ID: Contact me offlist if you're going - I'm thinking of attending. Stephen _________________________________________________________________ Hotmail: Trusted email with powerful SPAM protection. http://clk.atdmt.com/GBL/go/201469227/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From josh at engagementlab.org Wed Feb 10 01:24:19 2010 From: josh at engagementlab.org (Josh Begley) Date: Tue, 9 Feb 2010 19:24:19 -0500 Subject: [Baypiggies] JOB OPENING: PART TIME PYTHON/WEB SOFTWARE DEVELOPER Message-ID: <1475b8dd1002091624w5e0c25acja7a4ef1f97c6902d@mail.gmail.com> JOB OPENING: PART TIME PYTHON/WEB SOFTWARE DEVELOPER ********************************************************************************** Organizational Overview Citizen Engagement Laboratory (CEL) is a nonprofit, nonpartisan organization that inspires citizen engagement through digital media and technology. ?We do this by managing campaigns, producing videos and other media, and developing software. Job Description Most of what we do centers around online advocacy, and we roll a lot of our own stuff to get the job done. ?We do most of our work in Python with the Django framework. We're looking for someone with Python experience and some background doing Django work to do things like: * Implementation of campaign web sites, specifically back-end development * Custom development to support campaign work * Running database queries, reports for staff and other organizations We anticipate 15-20 hours per week, possibly more if desired. ?Time flexibility is essential, as campaigns can require intensive work at night. The position will be compensated on an hourly basis, with rate depending on experience. Qualifications * General development background and knowledge * Strong python background * Experience / knowledge of Django * General web development skills: HTML, CSS, Javascript (we mostly use jQuery) * General knowledge of Unix systems, Apache, etc. ?(we use Ubuntu GNU/Linux) * General knowledge of relational databases (PostgreSQL / MySQL) Application Interested applicants should email jobs at engagementlab.org with a resume and cover letter explaining why you are the right one for the job.? Location in SF Bay Area is preferred. Applications will be accepted until Feb. 12, 2010. From jjinux at gmail.com Wed Feb 10 09:26:03 2010 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Wed, 10 Feb 2010 00:26:03 -0800 Subject: [Baypiggies] asynch design In-Reply-To: <9b97850f1002081114j3103f67el89c29457706f41c6@mail.gmail.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <20100204222706.GA7542@panix.com> <4B704436.9000509@noir.com> <9b97850f1002081114j3103f67el89c29457706f41c6@mail.gmail.com> Message-ID: On Mon, Feb 8, 2010 at 11:14 AM, Stephen Lacy wrote: > Here's a great (and recent) comparison of a whole bunch of asynchronous web > servers in Python: > > http://nichol.as/asynchronous-servers-in-python > > (via http://firendfeed.com/amitp) I also gave a talk on this, both at BayPiggies and PyWebSF. Here's the blog post and the slides: http://jjinux.blogspot.com/2009/12/python-concurrency.html In the talk, I give a lot of background, and I also cover modern frameworks based on Greenlets that allow you to write code that is asynchronous without forcing you to write in callback style. It's the best of both worlds. -jj -- In this life we cannot do great things. We can only do small things with great love. -- Mother Teresa http://jjinux.blogspot.com/ From jjinux at gmail.com Wed Feb 10 09:33:18 2010 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Wed, 10 Feb 2010 00:33:18 -0800 Subject: [Baypiggies] virtual synchrony In-Reply-To: <4B6B14B2.5030801@noir.com> References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> <4B6B127D.5030209@noir.com> <4B6B14B2.5030801@noir.com> Message-ID: On Thu, Feb 4, 2010 at 10:40 AM, K. Richard Pixley wrote: > I'd also be willing to present on virtual synchrony. ?It's an idea from the > 80's which largely predates threads and has never caught on for reasons I've > never understood. > > In a nutshell, virtual synchrony is a system based on the idea of "reliable > broadcast", except that the definition of "reliable" has been perverted. > ?Instead of declaring that all receivers receive a broadcast "at the same > time", (which isn't really possible in a 4d time space continuum except for > very coarse grained definitions of "same time"), virtual synchrony > guarantees that all receivers receive all messages in precisely the same > order, (or they cease to participate in the virtual synchrony). > > As a base mechanism for coordination, virtual synchrony can be used to build > simple operating system concepts like mutual exclusion, shared state, > reliable publish/subscribe mechanisms, etc. ?The usual application is for > distributed, asynchronous, parallel, fault tolerant, network based services. That sounds like Spread: http://www.spread.org/ Happy Hacking! -jj -- In this life we cannot do great things. We can only do small things with great love. -- Mother Teresa http://jjinux.blogspot.com/ From rich at noir.com Wed Feb 10 17:21:13 2010 From: rich at noir.com (K. Richard Pixley) Date: Wed, 10 Feb 2010 08:21:13 -0800 Subject: [Baypiggies] virtual synchrony In-Reply-To: References: <1f7713e31002022121m7ff7ddc0j927a00d3682bd6ab@mail.gmail.com> <4B69B385.2060808@noir.com> <4B6A3690.3050507@noir.com> <4B6B127D.5030209@noir.com> <4B6B14B2.5030801@noir.com> Message-ID: <4B72DCF9.8060306@noir.com> Shannon -jj Behrens wrote: > That sounds like Spread: http://www.spread.org Yup. Spread is one implementation of virtual synchrony. --rich From john_re at fastmail.us Wed Feb 10 19:14:39 2010 From: john_re at fastmail.us (giovanni_re) Date: Wed, 10 Feb 2010 10:14:39 -0800 Subject: [Baypiggies] BIL Ride Today/morrow LA Feb 10-12-14 Python GNU(Linux)/FreedomSWHWCulture VOIP BerkeleyTIP Message-ID: <1265825679.28365.1359311637@webmail.messagingengine.com> BIL is the unconference companion to TED, independently produced, this Fri-Sun Feb 12-14 in Los Angeles (Long Beach). (Starts Friday 10AM.) http://2010.bilconference.com http://bilconference.com I'm gonna try to go, lead a session about Howto have global voice meetings, one about GNU(Linux), maybe have an informal installfest. Maybe you'd like to join the installfest, & help bring GNU(Linux) to the BIL crowd. Maybe you'd like to listen to a talk. Maybe you'd like to present a talk. If you're interested in going, would like to save a little money & time, wouldn't mind contributing to gas & expenses, & would like a ride, leaving later today likely (Wednesday), or perhaps early tomorrow, Thursday, from the San Fran bay area, email me with your phone # & maybe we can arrange something. == There are 2 _opportunities_: 1) learn & spend time with similar minded people by attending other people's presentations/talks/discussions, 2) learn & spend time with similar minded people by creating your own presentations/talks/discussions. Cost for BIL: $20 total for 3 days, or free if you volunteer. You'll need: money for food, spare clean clothes, maybe a sleeping bag or blanket, & a computer would be rather useful. === New info about speakers & talks, from the website. "BIL Speaking Spaces There are three talk formats for BIL this year, the Fire Hose, JIL, and the Open Culture Space. The Fire Hose consists of 20 minutes talks, with no Q&A and strictly enforced time limits. JIL is a set of talks curated for a feminine perspective. Open Culture is a Flexible time format. A number of spaces will be available for breakouts and socializing. The talks are being held at the Museum of Latin American Art in Long Beach, CA. 628 Alamitos Avenue, Long Beach, CA 90802 http://www.molaa.org/" http://2010.bilconference.com/ http://2010.bilconference.com/2010/02/bilxoutside/ "BIL has been growing, and we?re transitioning away from keeping everyone in one room the whole time to encouraging people to strike out in small groups for adventures and exploration in the nearby community. This isn?t being organized centrally, but this page and SideWiki are going to be an area for people to organize themselves." Info about the BIL mailing list: http://groups.google.com/group/bil-conference "Description: BIL has been created as a free space for people with ideas to come together and share them. Our event is self-organizing, emergent, and anarchic. Nobody is in charge. If you want to come just show up. If you've got an idea to spread start talking. If someone is saying something good, stop and listen" From luca.pellicoro at gmail.com Wed Feb 10 20:17:25 2010 From: luca.pellicoro at gmail.com (Luca Pellicoro) Date: Wed, 10 Feb 2010 11:17:25 -0800 Subject: [Baypiggies] [Job Posting] QA Automation Lead Message-ID: *Greetings, I'm on the QA automation team at Playdom, testing social games with plenty of challenges and rewards. We are looking for an engineer lead for the team. Work has started on an automation infrastructure with CPython driving selenium in the backend, but other tools are considered. I've been following this list for a while and learned a lot. I'd love to work under one talented Python enthusiast. I can answer questions publicly if everyone benefits or off-list. See job description below, --Luca From: http://newton.newtonsoftware.com/career/JobIntroduction.action?clientId=4028f88c23ff1cb8012411dda6a8059e&id=4028f88b23ff1cef0124372d70de11d8&source= Position:* QA Automation Lead *Location:* Mountain View, CA *Job Id:* QAAL22010 *# of Openings:* 1 ------------------------------ Test Automation Lead Technical Skills desired: - Flash - HTML - Java - MySQL - PHP - Python - Selenium Experience desired: - Web application automation - Load testing - Defect tracking (JIRA, etc.) - Software Configuration Management Qualities desired: Attention to detail. Ability to balance long-term desires with short-term need. Ability to document the better process, but follow the one in place. Ability to document while implementing. Ability to document, period. Ability to decide what not to automate. Ability to focus on delivery and deadlines. Good communications skills, verbal and written. Job duties: Create test plans for automation based on manual testing test plans and product descriptions. Define and implement diverse automation architecture for Flash and HTML/Javascript web applications. Direct team members in the development of automated test cases. Identify appropriate automation development to be done in office vs by offshore team. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim at well.com Wed Feb 10 22:38:47 2010 From: jim at well.com (jim) Date: Wed, 10 Feb 2010 13:38:47 -0800 Subject: [Baypiggies] call for PyCon talks for February Message-ID: <1265837927.9573.15.camel@jim-laptop> My apologies for my last-minute work for our upcoming February bayPIGgies meeting. In the past we've had pre-PyCon talks. If this seems appealing, please pipe up and volunteer to present your talk or at least a summary of your talk, should you be giving one in PyCon, or volunteer to give some opinions or news. In the absence of pre-PyCon talks, I'll try to get someone to present something--there are a couple or three possible talks in the works. jim From nagappan at gmail.com Thu Feb 11 08:29:40 2010 From: nagappan at gmail.com (Nagappan Alagappan) Date: Wed, 10 Feb 2010 23:29:40 -0800 Subject: [Baypiggies] Announce: Linux Desktop Testing Project (LDTP) 2.0.3 released Message-ID: <9d0602eb1002102329m3cb2092fsae00fdce25eb7490@mail.gmail.com> Hello, About LDTP: Linux Desktop Testing Project is aimed at producing high quality test automation framework (using GNOME / Python) and cutting-edge tools that can be used to test Linux Desktop and improve it. It uses the Accessibility libraries to poke through the application's user interface. We strive to help in building a quality desktop. Changes in this release: Return always unicode string in gettextvalue, required to fix automated test script in VMware Workstation Fix ooldtp compatibility with LDTPv1 as reported by Mago [1] team Patch by James Tatum for getallstates compatible with hasstate function Fix bug b.g.o#608413 Fix Firefox preference accessing bug, reported by Aaron Yuan < yuanchong1986 at gmail.com> Download source: http://download.freedesktop.org/ldtp/2.x/2.0.x/ldtp-2.0.3.tar.gz Download RPM from http://download.opensuse.org/repositories/home:/anagappan:/ldtp2:/rpm/ Will schedule deb build in openSUSE build service tomorrow Documentation references: For detailed information on LDTP framework and latest updates visit http://ldtp.freedesktop.org For information on various APIs in LDTP including those added for this release can be got from http://ldtp.freedesktop.org/user-doc/index.html Report bugs - http://ldtp.freedesktop.org/wiki/Bugs To subscribe to LDTP mailing lists, visit http://ldtp.freedesktop.org/wiki/Mailing_20list IRC Channel - #ldtp on irc.freenode.net Thanks Nagappan [1] - http://mago.ubuntu.com -- Linux Desktop (GUI Application) Testing Project - http://ldtp.freedesktop.org http://nagappanal.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From cappy2112 at gmail.com Fri Feb 12 05:35:32 2010 From: cappy2112 at gmail.com (Tony Cappellini) Date: Thu, 11 Feb 2010 20:35:32 -0800 Subject: [Baypiggies] I'm looking for a someone to review "Progamming In Python 3:A Complete Introduction to the Python Language" Message-ID: <8249c4ac1002112035p7eaa1b70w34868d06e39ba8b9@mail.gmail.com> If you are interested in writing a short review for this book, http://www.informit.com/store/product.aspx?isbn=0137129297 please email me off-list. You can keep the book, if you finish the review. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From nagappan at gmail.com Fri Feb 12 07:36:24 2010 From: nagappan at gmail.com (Nagappan Alagappan) Date: Thu, 11 Feb 2010 22:36:24 -0800 Subject: [Baypiggies] Announce: Linux Desktop Testing Project (LDTP) 1.7.2 released Message-ID: <9d0602eb1002112236r4e829ed6vc457121cec0c2030@mail.gmail.com> Greetings all, We are proud to announce the release of LDTP 1.7.1. This release features number of important breakthroughs in LDTP as well as in the field of Test Automation. This release note covers a brief introduction on LDTP followed by the list of new features and major bug fixes which makes this new version of LDTP the best of the breed. Useful references have been included at the end of this article for those who wish to hack / use LDTP. About LDTP: Linux Desktop Testing Project is aimed at producing high quality test automation framework (C / Python) and cutting-edge tools that can be used to test Linux Desktop and improve it. It uses the Accessibility libraries to poke through the application's user interface. The framework also has tools to record test-cases based on user events in the interface of the application which is under testing. We strive to help in building a quality desktop. NOTE: *** This is a bug fix release only as requested by Sun QA team, we recommend to use LDTPv2 *** Changes in this release: Patch by James Tatum to fix Bug 606224 - Exceptions do not return unicode data Patch by Ara Pulido to fix Bug 603115 - Activate wnck function is not working Bug 603118 - wnck wrappers do not work with LDTP window names Bug 605650 - LDTP core dumps on OpenSolaris Bug 598947 - waittillguiexist() documentation should say if wildcards/regexes are allowed Patch by Adam Collard to fix Bug 599119 - checkrow action fails on table row Special thanks to James Tatum, Ara Pulido, Murray Cumming, Conny Cheng, Adam Collard Download source - http://download.freedesktop.org/ldtp/1.x/1.7.x/ldtp-1.7.2.tar.gz Download rpm - http://download.opensuse.org/repositories/home:/anagappan/ Will schedule deb build soon in openSUSE Build Service References: For detailed information on LDTP framework and latest updates visit http://ldtp.freedesktop.org For information on various APIs in LDTP including those added for this release can be got from http://ldtp.freedesktop.org/user-doc/index.html To subscribe to LDTP mailing lists, visit http://ldtp.freedesktop.org/wiki/Mailing_20list IRC Channel - #ldtp on irc.freenode.net Thanks Nagappan -- Linux Desktop (GUI Application) Testing Project - http://ldtp.freedesktop.org http://nagappanal.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim at well.com Fri Feb 12 23:05:16 2010 From: jim at well.com (jim) Date: Fri, 12 Feb 2010 14:05:16 -0800 Subject: [Baypiggies] I'm looking for a someone to review "Progamming In Python 3:A Complete Introduction to the Python Language" In-Reply-To: <8249c4ac1002112035p7eaa1b70w34868d06e39ba8b9@mail.gmail.com> References: <8249c4ac1002112035p7eaa1b70w34868d06e39ba8b9@mail.gmail.com> Message-ID: <1266012316.14595.13.camel@jim-laptop> Let us know if you find a reviewer. It looks like I might have a lot of free time in the weeks ahead, and I might be interested in providing a more timely review than last time. jim On Thu, 2010-02-11 at 20:35 -0800, Tony Cappellini wrote: > > If you are interested in writing a short review for this book, > > http://www.informit.com/store/product.aspx?isbn=0137129297 > > please email me off-list. > > You can keep the book, if you finish the review. > > Thanks > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies From cappy2112 at gmail.com Fri Feb 12 23:14:13 2010 From: cappy2112 at gmail.com (Tony Cappellini) Date: Fri, 12 Feb 2010 14:14:13 -0800 Subject: [Baypiggies] Found a reviewer, no more replies necessary. was ( I'm looking for a someone to review "Progamming In Python 3:) Message-ID: <8249c4ac1002121414n5c98307r4e9dbdac50aa92b@mail.gmail.com> I already have a reviewer, thanks. No more replies neccessary. On Fri, Feb 12, 2010 at 2:05 PM, jim wrote: > > > Let us know if you find a reviewer. > It looks like I might have a lot of free > time in the weeks ahead, and I might be > interested in providing a more timely > review than last time. > jim > > > > On Thu, 2010-02-11 at 20:35 -0800, Tony Cappellini wrote: > > > > If you are interested in writing a short review for this book, > > > > http://www.informit.com/store/product.aspx?isbn=0137129297 > > > > please email me off-list. > > > > You can keep the book, if you finish the review. > > > > Thanks > > > > _______________________________________________ > > Baypiggies mailing list > > Baypiggies at python.org > > To change your subscription options or unsubscribe: > > http://mail.python.org/mailman/listinfo/baypiggies > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From beau at open-source-staffing.com Mon Feb 15 07:58:46 2010 From: beau at open-source-staffing.com (Beau Gould (OSS)) Date: Mon, 15 Feb 2010 01:58:46 -0500 Subject: [Baypiggies] [JOB] Python/Django Developer, San Francisco | 50-90k Message-ID: Python/Django Developer, San Francisco | 50-90k This is a full time, on-site, salaried position located in San Francisco, CA (94107) paying $50,000 to $90,000 (dep on exp) + benefits. US Citizens, Green Card holders, EAD or CAN only please. Local candidates only. No telecommuting. Thank you. You are a San Francisco-area based web developer who has solid experience writing software and designing sites for the web. You may not necessarily be a designer, but you understand principles of good visual design. You can design web pages with HTML, JavaScript, and CSS, and are also comfortable writing code which queries databases, accesses web services, and processes data from the command line. Required experience: - proven web front-end experience with HTML/CSS and Adobe CS tools - 2+ years experience with Python and Django Other desirable experience: - Ruby and Java - Web mapping, GIS, cartography - Amazon Web Services - Postgres/PostGIS, GEOS/GDAL/OGR - Flex/Flash - Data Warehousing - RESTful API design - Linux administration To be considered, please submit your resume and salary requirements to beau[AT]open-source-staffing.com Thank you, Beau J. Gould ------------------ Open Source Staffing http://www.open-source-staffing.com http://www.facebook.com/beau.gould beau[AT]open-source-staffing.com From aahz at pythoncraft.com Mon Feb 15 16:40:08 2010 From: aahz at pythoncraft.com (Aahz) Date: Mon, 15 Feb 2010 07:40:08 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: References: Message-ID: <20100215154007.GA2545@panix.com> On Mon, Feb 15, 2010, Beau Gould (OSS) wrote: > > Python/Django Developer, San Francisco | 50-90k > > This is a full time, on-site, salaried position located in San > Francisco, CA (94107) paying $50,000 to $90,000 (dep on exp) + > benefits. US Citizens, Green Card holders, EAD or CAN only please. > Local candidates only. No telecommuting. Thank you. Maybe it's just me, but this seems like a low salary range. What do other people think? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. :-)" From glen at glenjarvis.com Mon Feb 15 17:33:49 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Mon, 15 Feb 2010 08:33:49 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <20100215154007.GA2545@panix.com> References: <20100215154007.GA2545@panix.com> Message-ID: It's actually within the 'average' medium of the bell curve... According to several sites that measure people's salaries against job titles... So, if they're looking for an average programmer (not entry level and not an expert), it's about right... Cheers, Glen On Mon, Feb 15, 2010 at 7:40 AM, Aahz wrote: > On Mon, Feb 15, 2010, Beau Gould (OSS) wrote: > > > > Python/Django Developer, San Francisco | 50-90k > > > > This is a full time, on-site, salaried position located in San > > Francisco, CA (94107) paying $50,000 to $90,000 (dep on exp) + > > benefits. US Citizens, Green Card holders, EAD or CAN only please. > > Local candidates only. No telecommuting. Thank you. > > Maybe it's just me, but this seems like a low salary range. What do > other people think? > -- > Aahz (aahz at pythoncraft.com) <*> > http://www.pythoncraft.com/ > > "At Resolver we've found it useful to short-circuit any doubt and just > refer to comments in code as 'lies'. :-)" > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich at noir.com Mon Feb 15 17:52:12 2010 From: rich at noir.com (K. Richard Pixley) Date: Mon, 15 Feb 2010 08:52:12 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <20100215154007.GA2545@panix.com> References: <20100215154007.GA2545@panix.com> Message-ID: <4B797BBC.7050609@noir.com> Aahz wrote: > Maybe it's just me, but this seems like a low salary range. What do > other people think? > It's low for me, but I think that just means I'm not who they're looking for. If they find anyone at that price, then it wasn't low. And if they find no one, then they'll figure out that it was low soon enough. I think it's prolly a good thing they put it in the ad. That should save some people some time. --rich From aahz at pythoncraft.com Mon Feb 15 17:56:36 2010 From: aahz at pythoncraft.com (Aahz) Date: Mon, 15 Feb 2010 08:56:36 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: References: <20100215154007.GA2545@panix.com> Message-ID: <20100215165636.GA27630@panix.com> On Mon, Feb 15, 2010, Glen Jarvis wrote: > On Mon, Feb 15, 2010 at 7:40 AM, Aahz wrote: >> On Mon, Feb 15, 2010, Beau Gould (OSS) wrote: >>> >>> Python/Django Developer, San Francisco | 50-90k >>> >>> This is a full time, on-site, salaried position located in San >>> Francisco, CA (94107) paying $50,000 to $90,000 (dep on exp) + >>> benefits. US Citizens, Green Card holders, EAD or CAN only please. >>> Local candidates only. No telecommuting. Thank you. >> >> Maybe it's just me, but this seems like a low salary range. What do >> other people think? > > It's actually within the 'average' medium of the bell curve... > According to several sites that measure people's salaries against job > titles... > > So, if they're looking for an average programmer (not entry level and > not an expert), it's about right... Let's put it this way: how many people with real (not just titular) two or more years of experience with Django would take $50K/year (keeping in mind that real Django experience implies Python, SQL, HTML/CSS, and JavaScript), given how competitive the current market is for those skills? Speaking as someone who six months ago was on the searching side and is now on the hiring side, I definitely think this range is on the low end. But I would be interested in hearing contrary opinions from people with experience in hiring. (Leave out special circumstances such as an academic instutition.) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. :-)" From tony at tcapp.com Mon Feb 15 17:59:10 2010 From: tony at tcapp.com (Tony Cappellini) Date: Mon, 15 Feb 2010 08:59:10 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <20100215154007.GA2545@panix.com> References: <20100215154007.GA2545@panix.com> Message-ID: <8249c4ac1002150859x614e5cd6h950a9e82caebfa58@mail.gmail.com> > > > > > This is a full time, on-site, salaried position located in San > > Francisco, CA (94107) paying $50,000 to $90,000 (dep on exp) + > > benefits. US Citizens, Green Card holders, EAD or CAN only please. > > Local candidates only. No telecommuting. Thank you. > > >>Maybe it's just me, but this seems like a low salary range. What do > >>other people think? > Based on positions I've interviewed for, I'd say it's low, but I haven't interviewed in this "downturn in the economy". However nothing is stated as to how much experience is required for this position. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeff.enderwick at gmail.com Mon Feb 15 18:36:13 2010 From: jeff.enderwick at gmail.com (Jeff Enderwick) Date: Mon, 15 Feb 2010 09:36:13 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <20100215154007.GA2545@panix.com> References: <20100215154007.GA2545@panix.com> Message-ID: Yeah, I was gonna say something, like maybe a job using threads wouldn't be so bad after all ;-). On Mon, Feb 15, 2010 at 7:40 AM, Aahz wrote: > On Mon, Feb 15, 2010, Beau Gould (OSS) wrote: > > > > Python/Django Developer, San Francisco | 50-90k > > > > This is a full time, on-site, salaried position located in San > > Francisco, CA (94107) paying $50,000 to $90,000 (dep on exp) + > > benefits. US Citizens, Green Card holders, EAD or CAN only please. > > Local candidates only. No telecommuting. Thank you. > > Maybe it's just me, but this seems like a low salary range. What do > other people think? > -- > Aahz (aahz at pythoncraft.com) <*> > http://www.pythoncraft.com/ > > "At Resolver we've found it useful to short-circuit any doubt and just > refer to comments in code as 'lies'. :-)" > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From glen at glenjarvis.com Mon Feb 15 18:39:12 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Mon, 15 Feb 2010 09:39:12 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <20100215165636.GA27630@panix.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> Message-ID: > > Let's put it this way: how many people with real (not just titular) two > or more years of experience with Django would take $50K/year (keeping in > mind that real Django experience implies Python, SQL, HTML/CSS, and > JavaScript), given how competitive the current market is for those > skills? Speaking as someone who six months ago was on the searching side > and is now on the hiring side, I definitely think this range is on the > low end. But I would be interested in hearing contrary opinions from > people with experience in hiring. > I definitely agree the low end of this scale is too low. When I said "average" I wasn't looking at the median of that range (I was looking at the higher end). The median is $70K which is $10K - $15K lower than the average I mentioned. Oh, and these numbers do take the current economy into consideration. I hadn't rechecked these stats, though, for six months.. Cheers, Glen -------------- next part -------------- An HTML attachment was scrubbed... URL: From voidref at gmail.com Mon Feb 15 19:12:49 2010 From: voidref at gmail.com (Alan Westbrook) Date: Mon, 15 Feb 2010 10:12:49 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> Message-ID: <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> In 1998 entry level C++ devs were worth at least $45k in Santa Cruz (an area notorious for paying less than the Bay Area). So $50k seems quite low. Especially given that it's in San Francisco. Alan On Mon, Feb 15, 2010 at 9:39 AM, Glen Jarvis wrote: >> Let's put it this way: how many people with real (not just titular) two >> or more years of experience with Django would take $50K/year (keeping in >> mind that real Django experience implies Python, SQL, HTML/CSS, and >> JavaScript), given how competitive the current market is for those >> skills? ?Speaking as someone who six months ago was on the searching side >> and is now on the hiring side, I definitely think this range is on the >> low end. ?But I would be interested in hearing contrary opinions from >> people with experience in hiring. > > I?definitely?agree the low end of this scale is too low. When I said > "average" I wasn't looking at the median of that range (I was looking at the > higher end). The median is $70K which is $10K - $15K lower than the average > I mentioned. Oh, and these numbers do take the current economy into > consideration. > I hadn't rechecked these stats, though, for six months.. > Cheers, > > Glen > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From sfseth at gmail.com Mon Feb 15 22:33:54 2010 From: sfseth at gmail.com (Seth Friedman) Date: Mon, 15 Feb 2010 13:33:54 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> Message-ID: <64e45fca1002151333meb692dbof42985e24a34f7c2@mail.gmail.com> My first reaction to salary ranges this wide are that the people doing the hiring are not practicing the most critical thinking ever carried out. With quite a bit of experience on both sides of this equation, $50k in san francisco is an entry level computer enthusiast with a high school degree, customer support role. If someone took a job with any substantial professional experience in programming for $50k in SF here, I would be suspicious about them. And no, I don't believe those salary statistics sites until they are transparent about their data sources / methodology. ~seth On Mon, Feb 15, 2010 at 10:12 AM, Alan Westbrook wrote: > In 1998 entry level C++ devs were worth at least $45k in Santa Cruz > (an area notorious for paying less than the Bay Area). > > So $50k seems quite low. Especially given that it's in San Francisco. > > Alan > > On Mon, Feb 15, 2010 at 9:39 AM, Glen Jarvis wrote: > >> Let's put it this way: how many people with real (not just titular) two > >> or more years of experience with Django would take $50K/year (keeping in > >> mind that real Django experience implies Python, SQL, HTML/CSS, and > >> JavaScript), given how competitive the current market is for those > >> skills? Speaking as someone who six months ago was on the searching > side > >> and is now on the hiring side, I definitely think this range is on the > >> low end. But I would be interested in hearing contrary opinions from > >> people with experience in hiring. > > > > I definitely agree the low end of this scale is too low. When I said > > "average" I wasn't looking at the median of that range (I was looking at > the > > higher end). The median is $70K which is $10K - $15K lower than the > average > > I mentioned. Oh, and these numbers do take the current economy into > > consideration. > > I hadn't rechecked these stats, though, for six months.. > > Cheers, > > > > Glen > > _______________________________________________ > > Baypiggies mailing list > > Baypiggies at python.org > > To change your subscription options or unsubscribe: > > http://mail.python.org/mailman/listinfo/baypiggies > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aappiah at yahoo.com Tue Feb 16 03:09:59 2010 From: aappiah at yahoo.com (Alfred Appiah) Date: Mon, 15 Feb 2010 18:09:59 -0800 (PST) Subject: [Baypiggies] How to find the constants availiabe for a given module Message-ID: <206806.35647.qm@web37002.mail.mud.yahoo.com> Hi, I am new to python and wanted to know if there a way to find out all the constants, functions and classes available for a given module. For example the string module supports the following constants and functions, string.letters, string.digits string.whitespace string.capwords() and the only way I have been able to find out the classes and functions available is by reading the python docs , I.E http://docs.python.org/library/string.html . Are there commands I can type in the interpreter to get this information also ? Cheers Alfred From voidref at gmail.com Tue Feb 16 03:12:18 2010 From: voidref at gmail.com (Alan Westbrook) Date: Mon, 15 Feb 2010 18:12:18 -0800 Subject: [Baypiggies] How to find the constants availiabe for a given module In-Reply-To: <206806.35647.qm@web37002.mail.mud.yahoo.com> References: <206806.35647.qm@web37002.mail.mud.yahoo.com> Message-ID: <5df1e9b01002151812m5ed59c21te058747baac6880a@mail.gmail.com> dir(string) might be what you are looking for. On Mon, Feb 15, 2010 at 6:09 PM, Alfred Appiah wrote: > Hi, > > ?I am new to python and wanted to know if there a way to find out all the constants, functions ?and classes ?available for a given module. > > For example the string module supports the following constants and functions, > > string.letters, > string.digits > string.whitespace > string.capwords() > > and the only way I have been able to find out the classes and ?functions available is by reading the python docs , I.E http://docs.python.org/library/string.html . Are there commands I can type in the interpreter to get this information also ? > > Cheers Alfred > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From voidref at gmail.com Tue Feb 16 03:12:53 2010 From: voidref at gmail.com (Alan Westbrook) Date: Mon, 15 Feb 2010 18:12:53 -0800 Subject: [Baypiggies] How to find the constants availiabe for a given module In-Reply-To: <5df1e9b01002151812m5ed59c21te058747baac6880a@mail.gmail.com> References: <206806.35647.qm@web37002.mail.mud.yahoo.com> <5df1e9b01002151812m5ed59c21te058747baac6880a@mail.gmail.com> Message-ID: <5df1e9b01002151812q1a64e719ob3d1c1e88b725cd@mail.gmail.com> Oh, also help(string) On Mon, Feb 15, 2010 at 6:12 PM, Alan Westbrook wrote: > dir(string) might be what you are looking for. > > On Mon, Feb 15, 2010 at 6:09 PM, Alfred Appiah wrote: >> Hi, >> >> ?I am new to python and wanted to know if there a way to find out all the constants, functions ?and classes ?available for a given module. >> >> For example the string module supports the following constants and functions, >> >> string.letters, >> string.digits >> string.whitespace >> string.capwords() >> >> and the only way I have been able to find out the classes and ?functions available is by reading the python docs , I.E http://docs.python.org/library/string.html . Are there commands I can type in the interpreter to get this information also ? >> >> Cheers Alfred >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> http://mail.python.org/mailman/listinfo/baypiggies >> > From aappiah at yahoo.com Tue Feb 16 03:29:46 2010 From: aappiah at yahoo.com (Alfred Appiah) Date: Mon, 15 Feb 2010 18:29:46 -0800 (PST) Subject: [Baypiggies] How to find the constants availiabe for a given module In-Reply-To: <5df1e9b01002151812q1a64e719ob3d1c1e88b725cd@mail.gmail.com> Message-ID: <243813.79536.qm@web37001.mail.mud.yahoo.com> Ahh ok I see why it was not working when I was running 'help(string)' / 'dir(string)' etc and getting the message below . "NameError: name 'string' is not defined"..I forgot to add "import string" works now . cheers Al --- On Mon, 2/15/10, Alan Westbrook wrote: > From: Alan Westbrook > Subject: Re: [Baypiggies] How to find the constants availiabe for a given module > To: aappiah at yahoo.com > Cc: baypiggies at python.org > Date: Monday, February 15, 2010, 6:12 PM > Oh, also > > help(string) > > On Mon, Feb 15, 2010 at 6:12 PM, Alan Westbrook > wrote: > > dir(string) might be what you are looking for. > > > > On Mon, Feb 15, 2010 at 6:09 PM, Alfred Appiah > wrote: > >> Hi, > >> > >> ?I am new to python and wanted to know if there a > way to find out all the constants, functions ?and classes > ?available for a given module. > >> > >> For example the string module supports the > following constants and functions, > >> > >> string.letters, > >> string.digits > >> string.whitespace > >> string.capwords() > >> > >> and the only way I have been able to find out the > classes and ?functions available is by reading the python > docs , I.E http://docs.python.org/library/string.html . Are there > commands I can type in the interpreter to get this > information also ? > >> > >> Cheers Alfred > >> _______________________________________________ > >> Baypiggies mailing list > >> Baypiggies at python.org > >> To change your subscription options or > unsubscribe: > >> http://mail.python.org/mailman/listinfo/baypiggies > >> > > > From akwright at mac.com Tue Feb 16 03:42:12 2010 From: akwright at mac.com (Kevin Wright) Date: Mon, 15 Feb 2010 18:42:12 -0800 Subject: [Baypiggies] How to find the constants availiabe for a given module In-Reply-To: <243813.79536.qm@web37001.mail.mud.yahoo.com> References: <243813.79536.qm@web37001.mail.mud.yahoo.com> Message-ID: BTW, You can perform the help() and dir() without importing: >>> help('string') >>> dir('string') you just need to quote the library you are looking for info on. --Kevin On Feb 15, 2010, at 6:29 PM, Alfred Appiah wrote: > Ahh ok I see why it was not working when I was running > 'help(string)' / 'dir(string)' etc and getting the message below . > > "NameError: name 'string' is not defined"..I forgot to add "import > string" > > works now . > cheers Al > > --- On Mon, 2/15/10, Alan Westbrook wrote: > >> From: Alan Westbrook >> Subject: Re: [Baypiggies] How to find the constants availiabe for a >> given module >> To: aappiah at yahoo.com >> Cc: baypiggies at python.org >> Date: Monday, February 15, 2010, 6:12 PM >> Oh, also >> >> help(string) >> >> On Mon, Feb 15, 2010 at 6:12 PM, Alan Westbrook >> wrote: >>> dir(string) might be what you are looking for. >>> >>> On Mon, Feb 15, 2010 at 6:09 PM, Alfred Appiah >> wrote: >>>> Hi, >>>> >>>> I am new to python and wanted to know if there a >> way to find out all the constants, functions and classes >> available for a given module. >>>> >>>> For example the string module supports the >> following constants and functions, >>>> >>>> string.letters, >>>> string.digits >>>> string.whitespace >>>> string.capwords() >>>> >>>> and the only way I have been able to find out the >> classes and functions available is by reading the python >> docs , I.E http://docs.python.org/library/string.html . Are there >> commands I can type in the interpreter to get this >> information also ? >>>> >>>> Cheers Alfred >>>> _______________________________________________ From aleax at google.com Tue Feb 16 05:11:58 2010 From: aleax at google.com (Alex Martelli) Date: Mon, 15 Feb 2010 20:11:58 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <64e45fca1002151333meb692dbof42985e24a34f7c2@mail.gmail.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> <64e45fca1002151333meb692dbof42985e24a34f7c2@mail.gmail.com> Message-ID: <55dc209b1002152011x4f8647fgd72572d4fbd18134@mail.gmail.com> I find that a somewhat reasonable calibration for salary ranges can be found at www.glassdoor.com -- far from perfect, but useful. For "software engineer" in the "San Jose Area" they mention a range of 80-117, median 99; funny enough, it's somewhat higher on a national basis (!). (To see more than a couple pages on that site you'll have to register, and [[theoretically-untraceably]] post your own info). The reason I find their numbers to offer "a reasonable calibration" is that the numbers they give seem a sensible match in cases where I've been able to check. Of course, compensation's usually a pretty complicated matter (base salary is key, sure, but what about benefits, bonuses, options and/or stock, ...?). One bit about the state of the economy wrt hiring prospects: I haven't seen a breakdown by professional field or region, but, nationally, today's problems seem to be affecting mostly people with around-median or lower salaries -- around the $100k salary mark, unemployment is pretty stable wrt 2-3 years ago, and around the $150k mark, it's actually _lower_ than 2-3 years ago (!). Anecdotally, I just got an email from Van Linden about the forthcoming Pycon in Atlanta (this coming weekend), and the number and intensity of companies trying to hire great Python developers at that venue is scary (from the point of view of one such company -- Google/YouTube, at whose hiring-focused booth I'll try to be as often as I can, for example;-) // promising (from the point of view of great Python developers looking for a new gig;-). If you haven't signed up for Pycon, it's not too late -- hop on the "red eye" direct SF - Atlanta flight, sign up at the desk, and enjoy a weekend of great technical and social/networking time (including both formal and less formal job-hunters). If you can't make it to Pycon, but would still like to try your hand at a Google or YouTube interview, why not mail me privately with your resume? Next week (once I'm back from Pycon;-) I'll be delighted to try to help you (I'm biased, of course, since I'm a long-time, happy googler, _and_ I can get incentives for referring candidates who are strong [[and lucky]] enough to navigate all the way through our incredibly-demanding selection process -- but, mostly, I'm biased because I *crave* to keep hiring such candidates to be my colleagues!-). Alex On Mon, Feb 15, 2010 at 1:33 PM, Seth Friedman wrote: > My first reaction to salary ranges this wide are that the people doing the > hiring are not practicing the most critical thinking ever carried out. > > With quite a bit of experience on both sides of this equation, $50k in san > francisco is an entry level computer enthusiast with a high school degree, > customer support role.? If someone took a job with any substantial > professional experience in programming for $50k in SF here, I would be > suspicious about them. > > And no, I don't believe those salary statistics sites until they are > transparent about their data sources / methodology. > > ~seth > > On Mon, Feb 15, 2010 at 10:12 AM, Alan Westbrook wrote: >> >> In 1998 entry level C++ devs were worth at least $45k in Santa Cruz >> (an area notorious for paying less than the Bay Area). >> >> So $50k seems quite low. Especially given that it's in San Francisco. >> >> Alan >> >> On Mon, Feb 15, 2010 at 9:39 AM, Glen Jarvis wrote: >> >> Let's put it this way: how many people with real (not just titular) two >> >> or more years of experience with Django would take $50K/year (keeping >> >> in >> >> mind that real Django experience implies Python, SQL, HTML/CSS, and >> >> JavaScript), given how competitive the current market is for those >> >> skills? ?Speaking as someone who six months ago was on the searching >> >> side >> >> and is now on the hiring side, I definitely think this range is on the >> >> low end. ?But I would be interested in hearing contrary opinions from >> >> people with experience in hiring. >> > >> > I?definitely?agree the low end of this scale is too low. When I said >> > "average" I wasn't looking at the median of that range (I was looking at >> > the >> > higher end). The median is $70K which is $10K - $15K lower than the >> > average >> > I mentioned. Oh, and these numbers do take the current economy into >> > consideration. >> > I hadn't rechecked these stats, though, for six months.. >> > Cheers, >> > >> > Glen >> > _______________________________________________ >> > Baypiggies mailing list >> > Baypiggies at python.org >> > To change your subscription options or unsubscribe: >> > http://mail.python.org/mailman/listinfo/baypiggies >> > >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> http://mail.python.org/mailman/listinfo/baypiggies > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From joelvanderkwaak at yahoo.com Tue Feb 16 05:18:42 2010 From: joelvanderkwaak at yahoo.com (Joel VanderKwaak) Date: Mon, 15 Feb 2010 20:18:42 -0800 (PST) Subject: [Baypiggies] Salary ranges In-Reply-To: <55dc209b1002152011x4f8647fgd72572d4fbd18134@mail.gmail.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> <64e45fca1002151333meb692dbof42985e24a34f7c2@mail.gmail.com> <55dc209b1002152011x4f8647fgd72572d4fbd18134@mail.gmail.com> Message-ID: <264674.56305.qm@web50703.mail.re2.yahoo.com> Inquiring minds: For the record, Alex participated in my (unsuccessful) Google interview, and he's both sensible and a gentleman. No, I don't get a kickback for saying that, nor another interview ;) Joel ________________________________ From: Alex Martelli To: Seth Friedman Cc: baypiggies at python.org Sent: Mon, February 15, 2010 8:11:58 PM Subject: Re: [Baypiggies] Salary ranges ...snip.... If you can't make it to Pycon, but would still like to try your hand at a Google or YouTube interview, why not mail me privately with your resume? Next week (once I'm back from Pycon;-) I'll be delighted to try to help you (I'm biased, of course, since I'm a long-time, happy googler, _and_ I can get incentives for referring candidates who are strong [[and lucky]] enough to navigate all the way through our incredibly-demanding selection process -- but, mostly, I'm biased because I *crave* to keep hiring such candidates to be my colleagues!-). Alex ... snip... -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.salkever at gmail.com Tue Feb 16 05:51:02 2010 From: alex.salkever at gmail.com (Alex Salkever) Date: Mon, 15 Feb 2010 20:51:02 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <264674.56305.qm@web50703.mail.re2.yahoo.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> <64e45fca1002151333meb692dbof42985e24a34f7c2@mail.gmail.com> <55dc209b1002152011x4f8647fgd72572d4fbd18134@mail.gmail.com> <264674.56305.qm@web50703.mail.re2.yahoo.com> Message-ID: <11377cec1002152051p792ad603y66980cb5e79dc0a7@mail.gmail.com> Am not a coder but have done some hiring. I'll put it these way. People with 2 years of Rails (analogous to Django, roughly, right?) experience are *starting* at 80k in my world. For experienced devs, the range is closer to $100k - $125k. On Mon, Feb 15, 2010 at 8:18 PM, Joel VanderKwaak wrote: > Inquiring minds: > > For the record, Alex participated in my (unsuccessful) Google interview, > and he's both sensible and a gentleman. No, I don't get a kickback for > saying that, nor another interview ;) > > Joel > > ------------------------------ > *From:* Alex Martelli > *To:* Seth Friedman > *Cc:* baypiggies at python.org > *Sent:* Mon, February 15, 2010 8:11:58 PM > *Subject:* Re: [Baypiggies] Salary ranges > > ...snip.... > > > If you can't make it to Pycon, but would still like to try your hand > at a Google or YouTube interview, why not mail me privately with your > resume? Next week (once I'm back from Pycon;-) I'll be delighted to > try to help you (I'm biased, of course, since I'm a long-time, happy > googler, _and_ I can get incentives for referring candidates who are > strong [[and lucky]] enough to navigate all the way through our > incredibly-demanding selection process -- but, mostly, I'm biased > because I *crave* to keep hiring such candidates to be my > colleagues!-). > > > Alex > > ... snip... > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -- Alex Salkever p:415.503.9035 http://twitter.com/AlexSalkever -------------- next part -------------- An HTML attachment was scrubbed... URL: From beau at open-source-staffing.com Tue Feb 16 05:55:31 2010 From: beau at open-source-staffing.com (Beau Gould (OSS)) Date: Mon, 15 Feb 2010 23:55:31 -0500 Subject: [Baypiggies] [JOB] Python/Django Developer, San Francisco | 50-90k Message-ID: <1A1EA1A899AF47F99CC8F59B8DAB4382@EMACHINE> Hey Piggies, Aahz brought it to my attention after I submitted the job to Python job board for approval that there was a discussion going on here about the salary range for this position. I'm curious why you all chat about the low end of the range. The person that gets this job will probably command a 90-100k salary despite the 90k cap. Perhaps a more junior person with 6 months Django will get the job for 65k. I never consider the required experience written in stone and you shouldn't either. I have yet to receive one candidate thus far. I'm hoping tomorrow (after the holiday) will generate some candidates. Thank you, Beau J. Gould ------------------ Open Source Staffing http://www.open-source-staffing.com groups.yahoo.com/group/pythonzopejobs beau[AT]open-source-staffing.com -----Original Message----- From: Beau Gould (OSS) [mailto:beau at open-source-staffing.com] Sent: Monday, February 15, 2010 1:59 AM To: 'baypiggies at python.org' Subject: [JOB] Python/Django Developer, San Francisco | 50-90k Python/Django Developer, San Francisco | 50-90k This is a full time, on-site, salaried position located in San Francisco, CA (94107) paying $50,000 to $90,000 (dep on exp) + benefits. US Citizens, Green Card holders, EAD or CAN only please. Local candidates only. No telecommuting. Thank you. You are a San Francisco-area based web developer who has solid experience writing software and designing sites for the web. You may not necessarily be a designer, but you understand principles of good visual design. You can design web pages with HTML, JavaScript, and CSS, and are also comfortable writing code which queries databases, accesses web services, and processes data from the command line. Required experience: - proven web front-end experience with HTML/CSS and Adobe CS tools - 2+ years experience with Python and Django Other desirable experience: - Ruby and Java - Web mapping, GIS, cartography - Amazon Web Services - Postgres/PostGIS, GEOS/GDAL/OGR - Flex/Flash - Data Warehousing - RESTful API design - Linux administration To be considered, please submit your resume and salary requirements to beau[AT]open-source-staffing.com Thank you, Beau J. Gould ------------------ Open Source Staffing http://www.open-source-staffing.com http://www.facebook.com/beau.gould beau[AT]open-source-staffing.com From sganguly at yahoo.com Tue Feb 16 06:05:35 2010 From: sganguly at yahoo.com (Sukanta ganguly) Date: Mon, 15 Feb 2010 21:05:35 -0800 (PST) Subject: [Baypiggies] Salary ranges In-Reply-To: <11377cec1002152051p792ad603y66980cb5e79dc0a7@mail.gmail.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> <64e45fca1002151333meb692dbof42985e24a34f7c2@mail.gmail.com> <55dc209b1002152011x4f8647fgd72572d4fbd18134@mail.gmail.com> <264674.56305.qm@web50703.mail.re2.yahoo.com> <11377cec1002152051p792ad603y66980cb5e79dc0a7@mail.gmail.com> Message-ID: <893961.93976.qm@web112307.mail.gq1.yahoo.com> Hi, I am looking for a strong Java developer with experience i design/architecture and development experience. This is a senior Java developer to an architect's role. Please contact me if anybody is interested or please forward to people that you know who would be interested. Thanks SG ________________________________ From: Alex Salkever To: Joel VanderKwaak Cc: baypiggies at python.org Sent: Mon, February 15, 2010 8:51:02 PM Subject: Re: [Baypiggies] Salary ranges Am not a coder but have done some hiring. I'll put it these way. People with 2 years of Rails (analogous to Django, roughly, right?) experience are *starting* at 80k in my world. For experienced devs, the range is closer to $100k - $125k. On Mon, Feb 15, 2010 at 8:18 PM, Joel VanderKwaak wrote: Inquiring minds: > >For the record, Alex participated in my (unsuccessful) Google interview, and he's both sensible and a gentleman. No, I don't get a kickback for saying that, nor another interview ;) > >Joel > > > > ________________________________ From: Alex Martelli >To: Seth Friedman >Cc: baypiggies at python.org >Sent: Mon, February 15, 2010 8:11:58 PM >Subject: Re: [Baypiggies] Salary ranges > >>...snip.... > > >If you can't make it to Pycon, but would still like to try your hand >at a Google or YouTube interview, why not mail me privately with your >resume? Next week (once I'm back from Pycon;-) I'll be delighted to >>try to help you (I'm biased, of course, since I'm a long-time, happy >googler, _and_ I can get incentives for referring candidates who are >strong [[and lucky]] enough to navigate all the way through our >incredibly-demanding selection process -- but, mostly, I'm biased >>because I *crave* to keep hiring such candidates to be my >colleagues!-). > > >Alex > >... snip... > >_______________________________________________ >>Baypiggies mailing list >Baypiggies at python.org >>To change your subscription options or unsubscribe: >http://mail.python.org/mailman/listinfo/baypiggies > -- Alex Salkever p:415.503.9035 http://twitter.com/AlexSalkever -------------- next part -------------- An HTML attachment was scrubbed... URL: From voidref at gmail.com Tue Feb 16 07:30:53 2010 From: voidref at gmail.com (Alan Westbrook) Date: Mon, 15 Feb 2010 22:30:53 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <893961.93976.qm@web112307.mail.gq1.yahoo.com> References: <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> <64e45fca1002151333meb692dbof42985e24a34f7c2@mail.gmail.com> <55dc209b1002152011x4f8647fgd72572d4fbd18134@mail.gmail.com> <264674.56305.qm@web50703.mail.re2.yahoo.com> <11377cec1002152051p792ad603y66980cb5e79dc0a7@mail.gmail.com> <893961.93976.qm@web112307.mail.gq1.yahoo.com> Message-ID: <5df1e9b01002152230y4d9c577qd08f02944ad29bba@mail.gmail.com> Hi, this is a Python dev list, please do not post offtopic. Thanks! On Mon, Feb 15, 2010 at 9:05 PM, Sukanta ganguly wrote: > Hi, > ?? I am looking for a strong Java developer with experience i > design/architecture and development experience. This is a senior Java > developer to an architect's role. Please contact me if anybody is interested > or please forward to people that you know who would be interested. > > Thanks > SG > > ________________________________ > From: Alex Salkever > To: Joel VanderKwaak > Cc: baypiggies at python.org > Sent: Mon, February 15, 2010 8:51:02 PM > Subject: Re: [Baypiggies] Salary ranges > > Am not a coder but have done some hiring. I'll put it these way. People with > 2 years of Rails (analogous to Django, roughly, right?) experience are > *starting* at 80k in my world. For experienced devs, the range is closer to > $100k - $125k. > > On Mon, Feb 15, 2010 at 8:18 PM, Joel VanderKwaak > wrote: >> >> Inquiring minds: >> >> For the record, Alex participated in my (unsuccessful) Google interview, >> and he's both sensible and a gentleman. No, I don't get a kickback for >> saying that, nor another interview ;) >> >> Joel >> >> ________________________________ >> From: Alex Martelli >> To: Seth Friedman >> Cc: baypiggies at python.org >> Sent: Mon, February 15, 2010 8:11:58 PM >> Subject: Re: [Baypiggies] Salary ranges >> >> ...snip.... >> >> If you can't make it to Pycon, but would still like to try your hand >> at a Google or YouTube interview, why not mail me privately with your >> resume?? Next week (once I'm back from Pycon;-) I'll be delighted to >> try to help you (I'm biased, of course, since I'm a long-time, happy >> googler, _and_ I can get incentives for referring candidates who are >> strong [[and lucky]] enough to navigate all the way through our >> incredibly-demanding selection process -- but, mostly, I'm biased >> because I *crave* to keep hiring such candidates to be my >> colleagues!-). >> >> >> Alex >> >> ... snip... >> >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> http://mail.python.org/mailman/listinfo/baypiggies > > > > -- > Alex Salkever > p:415.503.9035 > http://twitter.com/AlexSalkever > > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From fenn at SDF.LONESTAR.ORG Tue Feb 16 07:28:07 2010 From: fenn at SDF.LONESTAR.ORG (ben lipkowitz) Date: Tue, 16 Feb 2010 06:28:07 +0000 (UTC) Subject: [Baypiggies] Salary ranges In-Reply-To: <893961.93976.qm@web112307.mail.gq1.yahoo.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> <64e45fca1002151333meb692dbof42985e24a34f7c2@mail.gmail.com> <55dc209b1002152011x4f8647fgd72572d4fbd18134@mail.gmail.com> <264674.56305.qm@web50703.mail.re2.yahoo.com> <11377cec1002152051p792ad603y66980cb5e79dc0a7@mail.gmail.com> <893961.93976.qm@web112307.mail.gq1.yahoo.com> Message-ID: On Mon, 15 Feb 2010, Sukanta ganguly wrote: > Hi, > I am looking for a strong Java developer with experience i > design/architecture and development experience. This is a senior Java > developer to an architect's role. Please contact me if anybody is > interested or please forward to people that you know who would be > interested. forwarded to bayjiggies at java.com you're welcome. From kbighorse at yahoo.com Tue Feb 16 08:24:22 2010 From: kbighorse at yahoo.com (Kimball Bighorse) Date: Mon, 15 Feb 2010 23:24:22 -0800 (PST) Subject: [Baypiggies] Resizing an image? Message-ID: <668173.93124.qm@web82302.mail.mud.yahoo.com> Looking at ImageMagick with PythonMagickWand for resizing an image, and getting an architecture mismatch I'd rather avoid. Familiar with FreeImage from Rails and heard about PIL for Python, any experience with either or recommend the ImageMagick approach? Kimball From keith at dartworks.biz Tue Feb 16 08:50:14 2010 From: keith at dartworks.biz (Keith Dart) Date: Mon, 15 Feb 2010 23:50:14 -0800 Subject: [Baypiggies] Resizing an image? In-Reply-To: <668173.93124.qm@web82302.mail.mud.yahoo.com> References: <668173.93124.qm@web82302.mail.mud.yahoo.com> Message-ID: <20100215235014.65df1f01@dartworks.biz> === On Mon, 02/15, Kimball Bighorse wrote: === > Looking at ImageMagick with PythonMagickWand for resizing an image, > and getting an architecture mismatch I'd rather avoid. Familiar with > FreeImage from Rails and heard about PIL for Python, any experience > with either or recommend the ImageMagick approach? === For something basic like that PIL should work fine. -- Keith Dart -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: 19017044 ===================================================================== From ideamonk at gmail.com Tue Feb 16 13:46:40 2010 From: ideamonk at gmail.com (Abhishek Mishra) Date: Tue, 16 Feb 2010 18:16:40 +0530 Subject: [Baypiggies] Resizing an image? In-Reply-To: <20100215235014.65df1f01@dartworks.biz> References: <668173.93124.qm@web82302.mail.mud.yahoo.com> <20100215235014.65df1f01@dartworks.biz> Message-ID: <64160c71002160446n2cfa15cem93d6a6722286bfde@mail.gmail.com> I would recommend PIL. Its as easy as fooImageObject.resize( (fooWidth, fooHeight) ) On Tue, Feb 16, 2010 at 1:20 PM, Keith Dart wrote: > === On Mon, 02/15, Kimball Bighorse wrote: === >> Looking at ImageMagick with PythonMagickWand for resizing an image, From bdbaddog at gmail.com Tue Feb 16 20:28:28 2010 From: bdbaddog at gmail.com (William Deegan) Date: Tue, 16 Feb 2010 11:28:28 -0800 Subject: [Baypiggies] python bittorent client library? Message-ID: <8540148a1002161128qcf859f7jd1205e52663f3ddb@mail.gmail.com> Greetings, I need to push a big bunch of data around to a bunch of cluster nodes and run some install scripts once the blob of data has been fully recieved. I was thinking of using RSS + Bittorrent. Are there any python bittorrent client libraries? Here's the flow I'm thinking of: 1) Publish via rss that a new bundle is available 2) each cluster node's polling the rss file, when it detects a new update 3) start bittorrent client to pull data 4) once data is fully transferred, run an install script 5) once 2x data received or some max time, shut down bt client. -Bill From brent.tubbs at gmail.com Tue Feb 16 20:47:02 2010 From: brent.tubbs at gmail.com (Brent Tubbs) Date: Tue, 16 Feb 2010 11:47:02 -0800 Subject: [Baypiggies] python bittorent client library? In-Reply-To: <8540148a1002161128qcf859f7jd1205e52663f3ddb@mail.gmail.com> References: <8540148a1002161128qcf859f7jd1205e52663f3ddb@mail.gmail.com> Message-ID: <769bb4301002161147j6c9b0746v9625d10f4d38b6e5@mail.gmail.com> A little googling led me to Pyroscope (http://code.google.com/p/pyroscope/), which may do what you need. Wasn't the original bittorrent implementation written in python? I have a vague memory to that effect. Brent On Tue, Feb 16, 2010 at 11:28 AM, William Deegan wrote: > Greetings, > > I need to push a big bunch of data around to a bunch of cluster nodes > and run some install scripts once the blob of data has been fully > recieved. > I was thinking of using RSS + Bittorrent. > > Are there any python bittorrent client libraries? > > Here's the flow I'm thinking of: > 1) Publish via rss that a new bundle is available > 2) each cluster node's polling the rss file, when it detects a new update > 3) start bittorrent client to pull data > 4) once data is fully transferred, run an install script > 5) once 2x data received or some max time, shut down bt client. > > -Bill > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy.r.fishman at gmail.com Tue Feb 16 21:10:39 2010 From: jeremy.r.fishman at gmail.com (Jeremy Fishman) Date: Tue, 16 Feb 2010 12:10:39 -0800 Subject: [Baypiggies] python bittorent client library? In-Reply-To: <8540148a1002161128qcf859f7jd1205e52663f3ddb@mail.gmail.com> References: <8540148a1002161128qcf859f7jd1205e52663f3ddb@mail.gmail.com> Message-ID: <1f7713e31002161210v3bc6d689r792798c017718f28@mail.gmail.com> The main BitTorrent client has always had a Python reference implementation and a Python-based tracker. Now that's it's been closed off within uTorrent it's hard to find these resources Rather than writing your own Python client, I suggest you look into Transmission (http://www.transmissionbt.com/). You can set up Transmission * as a headless system service ( http://trac.transmissionbt.com/wiki/HeadlessUsage) * setup the config to watch a directory for torrent files ( http://trac.transmissionbt.com/wiki/ConfigurationParameters) * OR - developer version has RSS feed capabilities, plus there's an example Python script for this * OR - use the RPC support to remotely add torrents * run a script on torrent completion (e.g. http://trac.transmissionbt.com/wiki/Scripts/EmailNotifier) There appears not to be great support for the last task there; the example script is a cron job that watches for completed files. There may be a real feature to support this, but I haven't looked into it. - Jeremy -- Jeremy R. Fishman Software Developer, Quantcast jeremy.r.fishman at gmail.com On Tue, Feb 16, 2010 at 11:28 AM, William Deegan wrote: > Greetings, > > I need to push a big bunch of data around to a bunch of cluster nodes > and run some install scripts once the blob of data has been fully > recieved. > I was thinking of using RSS + Bittorrent. > > Are there any python bittorrent client libraries? > > Here's the flow I'm thinking of: > 1) Publish via rss that a new bundle is available > 2) each cluster node's polling the rss file, when it detects a new update > 3) start bittorrent client to pull data > 4) once data is fully transferred, run an install script > 5) once 2x data received or some max time, shut down bt client. > > -Bill > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p at ulmcnett.com Tue Feb 16 21:11:46 2010 From: p at ulmcnett.com (Paul McNett) Date: Tue, 16 Feb 2010 12:11:46 -0800 Subject: [Baypiggies] Resizing an image? In-Reply-To: <64160c71002160446n2cfa15cem93d6a6722286bfde@mail.gmail.com> References: <668173.93124.qm@web82302.mail.mud.yahoo.com> <20100215235014.65df1f01@dartworks.biz> <64160c71002160446n2cfa15cem93d6a6722286bfde@mail.gmail.com> Message-ID: <4B7AFC02.7080904@ulmcnett.com> On 2/16/10 4:46 AM, Abhishek Mishra wrote: > I would recommend PIL. > > Its as easy as fooImageObject.resize( (fooWidth, fooHeight) ) Seconded. The API is really easy. Once your needs get more complex though, like needing alpha blending or font anti-aliasing, you are going to be going out of scope WRT PIL's design. Also, I still can't figure out how to compile PIL on Mac, which disables a small feature of one of my applications when I deploy to that platform. Paul From tnm800 at gmail.com Tue Feb 16 21:16:17 2010 From: tnm800 at gmail.com (Ted Nyman) Date: Tue, 16 Feb 2010 12:16:17 -0800 Subject: [Baypiggies] Resizing an image? In-Reply-To: <4B7AFC02.7080904@ulmcnett.com> References: <668173.93124.qm@web82302.mail.mud.yahoo.com> <20100215235014.65df1f01@dartworks.biz> <64160c71002160446n2cfa15cem93d6a6722286bfde@mail.gmail.com> <4B7AFC02.7080904@ulmcnett.com> Message-ID: <3fdfa9d61002161216o339cc899wd893ab340a120c18@mail.gmail.com> Getting PIL to work on OS X is notoriously problematic. Have you checked out this SO post? http://stackoverflow.com/questions/1438270/installing-python-imaging-library-pil-on-snow-leopard-with-updated-python-2-6-2 -Ted On Tue, Feb 16, 2010 at 12:11 PM, Paul McNett

wrote: > On 2/16/10 4:46 AM, Abhishek Mishra wrote: > >> I would recommend PIL. >> >> Its as easy as fooImageObject.resize( (fooWidth, fooHeight) ) >> > > Seconded. The API is really easy. Once your needs get more complex though, > like needing alpha blending or font anti-aliasing, you are going to be going > out of scope WRT PIL's design. > > Also, I still can't figure out how to compile PIL on Mac, which disables a > small feature of one of my applications when I deploy to that platform. > > Paul > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew at atoulou.se Tue Feb 16 21:28:44 2010 From: andrew at atoulou.se (Andrew Akira Toulouse) Date: Tue, 16 Feb 2010 12:28:44 -0800 Subject: [Baypiggies] python bittorent client library? In-Reply-To: <8540148a1002161128qcf859f7jd1205e52663f3ddb@mail.gmail.com> References: <8540148a1002161128qcf859f7jd1205e52663f3ddb@mail.gmail.com> Message-ID: Having tried to do something like this in the past, it's probably easiest to go with Transmission (as previously suggested) or rtorrent and use their RPC interfaces. On Tue, Feb 16, 2010 at 11:28 AM, William Deegan wrote: > Greetings, > > I need to push a big bunch of data around to a bunch of cluster nodes > and run some install scripts once the blob of data has been fully > recieved. > I was thinking of using RSS + Bittorrent. > > Are there any python bittorrent client libraries? > > Here's the flow I'm thinking of: > 1) Publish via rss that a new bundle is available > 2) each cluster node's polling the rss file, when it detects a new update > 3) start bittorrent client to pull data > 4) once data is fully transferred, run an install script > 5) once 2x data received or some max time, shut down bt client. > > -Bill > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From p at ulmcnett.com Tue Feb 16 21:29:44 2010 From: p at ulmcnett.com (Paul McNett) Date: Tue, 16 Feb 2010 12:29:44 -0800 Subject: [Baypiggies] Resizing an image? In-Reply-To: <3fdfa9d61002161216o339cc899wd893ab340a120c18@mail.gmail.com> References: <668173.93124.qm@web82302.mail.mud.yahoo.com> <20100215235014.65df1f01@dartworks.biz> <64160c71002160446n2cfa15cem93d6a6722286bfde@mail.gmail.com> <4B7AFC02.7080904@ulmcnett.com> <3fdfa9d61002161216o339cc899wd893ab340a120c18@mail.gmail.com> Message-ID: <4B7B0038.4090806@ulmcnett.com> On 2/16/10 12:16 PM, Ted Nyman wrote: > Getting PIL to work on OS X is notoriously problematic. Have you checked > out this SO post? > > http://stackoverflow.com/questions/1438270/installing-python-imaging-library-pil-on-snow-leopard-with-updated-python-2-6-2 Thanks for that link. I've filed it for further review. I'm currently still on plain-old-leopard though, as most of my clients are too and I'm afraid that if I start building my apps on SL, there will be problems distributing to L. But maybe the solution in this article would work for 2.5, too. Paul From nad at acm.org Tue Feb 16 22:40:12 2010 From: nad at acm.org (Ned Deily) Date: Tue, 16 Feb 2010 13:40:12 -0800 Subject: [Baypiggies] Resizing an image? References: <668173.93124.qm@web82302.mail.mud.yahoo.com> <20100215235014.65df1f01@dartworks.biz> <64160c71002160446n2cfa15cem93d6a6722286bfde@mail.gmail.com> <4B7AFC02.7080904@ulmcnett.com> <3fdfa9d61002161216o339cc899wd893ab340a120c18@mail.gmail.com> <4B7B0038.4090806@ulmcnett.com> Message-ID: In article <4B7B0038.4090806 at ulmcnett.com>, Paul McNett

wrote: > On 2/16/10 12:16 PM, Ted Nyman wrote: > > Getting PIL to work on OS X is notoriously problematic. Have you checked > > out this SO post? > > > > http://stackoverflow.com/questions/1438270/installing-python-imaging-library > > -pil-on-snow-leopard-with-updated-python-2-6-2 > > Thanks for that link. I've filed it for further review. I'm currently still > on > plain-old-leopard though, as most of my clients are too and I'm afraid that > if I > start building my apps on SL, there will be problems distributing to L. But > maybe the > solution in this article would work for 2.5, too. If you need to install PIL for a particular OS X system and python version, the simplest way by far is to install everything from MacPorts: # download and install base MacPorts infrastructure if necessary, then: sudo port selfupdate # ensure port files are up-to-date sudo port install py25-pil (or sudo port install py26-pil) That will build and install the appropriate python, PIL, and any necessary dependent libraries which is the tricky part. You can also just use MacPorts to install the libraries (freetype, jpeg, et al) and do a source install of PIL to an existing Python, either an Apple-supplied one or a python.org installer one, but you do need to be careful to build the necessary missing libraries with all of the appropriate architectures. The easiest way to do that is add +universal to variants.conf in /opt/local/etc/macports before installing. If you are building an app for distribution to a mix of OS X systems, you should definitely stick to building on 10.5 rather than 10.6 for the time being if you can, especially if you are using py2app. There are still many more potential gotchas on 10.6, primarily as a result of the move to 64-bit as default. You can make it work and, over time, the rough edges will get fixed but, for now, sticking with 10.5 will undoubtedly be easier. -- Ned Deily, nad at acm.org From gracelaw at mac.com Tue Feb 16 21:49:35 2010 From: gracelaw at mac.com (Grace Law) Date: Tue, 16 Feb 2010 12:49:35 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> Message-ID: <480ED171-7523-491A-BF76-B08CD7703668@mac.com> Sorry for the late entry to this thread. IMHO, $50K is exploitation, even for an average developer, but many companies do that and think it is OK. Personally, I'd love to hire engineers away from these companies. The companies I have recruited for pay far more than that, even for an intern. Alex is right, glassdoor.com offers a better view. But salary is not the only thing. You have to work with a company that treats you well. If you are reading this thread and having a tough time finding a job, don't settle for a job just to have a job. Rather, improve your technical skills ( work on personal side projects, contribute to open source, attend the many meetups now available)... at the same time, improve your communications, interview, and puzzle solving skills... and get a darn good work ethics. You WILL find something decent with better pay. Companies are hunting for talents - and are willing to pay for them, at least mine are. Let me know if you need help. If I don't respond quickly, it's because I will be pretty busy at PyCon. Talk to you next week. Best, Grace Law Sr Technical Recruiter @ Lolapps and Sauce Labs gracelaw at mac.com 650-823-7236 cell http://www.linkedin.com/in/gracelaw1972 On Feb 15, 2010, at 10:12 AM, Alan Westbrook wrote: > In 1998 entry level C++ devs were worth at least $45k in Santa Cruz > (an area notorious for paying less than the Bay Area). > > So $50k seems quite low. Especially given that it's in San Francisco. > > Alan > > On Mon, Feb 15, 2010 at 9:39 AM, Glen Jarvis > wrote: >>> Let's put it this way: how many people with real (not just >>> titular) two >>> or more years of experience with Django would take $50K/year >>> (keeping in >>> mind that real Django experience implies Python, SQL, HTML/CSS, and >>> JavaScript), given how competitive the current market is for those >>> skills? Speaking as someone who six months ago was on the >>> searching side >>> and is now on the hiring side, I definitely think this range is >>> on the >>> low end. But I would be interested in hearing contrary opinions >>> from >>> people with experience in hiring. >> >> I definitely agree the low end of this scale is too low. When I said >> "average" I wasn't looking at the median of that range (I was >> looking at the >> higher end). The median is $70K which is $10K - $15K lower than >> the average >> I mentioned. Oh, and these numbers do take the current economy into >> consideration. >> I hadn't rechecked these stats, though, for six months.. >> Cheers, >> >> Glen >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> http://mail.python.org/mailman/listinfo/baypiggies >> > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at zachary.com Tue Feb 16 22:50:57 2010 From: david at zachary.com (David Creemer) Date: Tue, 16 Feb 2010 13:50:57 -0800 Subject: [Baypiggies] Resizing an image? In-Reply-To: References: <668173.93124.qm@web82302.mail.mud.yahoo.com> <20100215235014.65df1f01@dartworks.biz> <64160c71002160446n2cfa15cem93d6a6722286bfde@mail.gmail.com> <4B7AFC02.7080904@ulmcnett.com> <3fdfa9d61002161216o339cc899wd893ab340a120c18@mail.gmail.com> <4B7B0038.4090806@ulmcnett.com> Message-ID: After struggling with PIL on Mac OS X for a while, I found the most reliable way to develop against it is to install a "headless" Linux instance under VirtualBox and sshfs-mount a shared file-system between systems. I edit via Mac OS, but run under Linux. Sad but true. The added advantage for me was the production deployment target was also Linux. -- David On Feb 16, 2010, at 1:40 PM, Ned Deily wrote: > If you need to install PIL for a particular OS X system and python > version, the simplest way by far is to install everything from MacPorts: From david at zachary.com Tue Feb 16 23:05:37 2010 From: david at zachary.com (David Creemer) Date: Tue, 16 Feb 2010 14:05:37 -0800 Subject: [Baypiggies] python bittorent client library? In-Reply-To: <8540148a1002161128qcf859f7jd1205e52663f3ddb@mail.gmail.com> References: <8540148a1002161128qcf859f7jd1205e52663f3ddb@mail.gmail.com> Message-ID: <1252005E-EB27-4A9F-A05D-25FC8C13308D@zachary.com> Twitter seems to have just released "Murder", which sounds exactly like what you want: http://github.com/lg/murder/ -- David On Feb 16, 2010, at 11:28 AM, William Deegan wrote: > Greetings, > > I need to push a big bunch of data around to a bunch of cluster nodes > and run some install scripts once the blob of data has been fully > recieved. > I was thinking of using RSS + Bittorrent. > > Are there any python bittorrent client libraries? > > Here's the flow I'm thinking of: > 1) Publish via rss that a new bundle is available > 2) each cluster node's polling the rss file, when it detects a new update > 3) start bittorrent client to pull data > 4) once data is fully transferred, run an install script > 5) once 2x data received or some max time, shut down bt client. > > -Bill > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies From hasan.diwan at gmail.com Tue Feb 16 23:05:47 2010 From: hasan.diwan at gmail.com (Hasan Diwan) Date: Tue, 16 Feb 2010 14:05:47 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <480ED171-7523-491A-BF76-B08CD7703668@mac.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> <480ED171-7523-491A-BF76-B08CD7703668@mac.com> Message-ID: <2cda2fc91002161405x45a1486bja8df77ec366c59e9@mail.gmail.com> On 16 February 2010 12:49, Grace Law wrote: > But salary is not the only thing. ?You have to work with a company that > treats you well. ?If you are reading this thread and having a tough time > finding a job, don't settle for a job just to have a job. ? Rather, improve > your technical skills ( work on personal side projects, contribute to open > source, attend the many meetups now available)... ?at the same time, improve > your communications, interview, and puzzle solving skills... and get a darn > good work ethics. ? You WILL find something decent with better pay. > Companies are hunting for talents - and are willing to pay for them, at > least mine are. Agreed with the skills improvement. I have found that the non-technical skills are more important (especially how you come across in an interview) and that, technical people including myself, tend to dismiss communications skills as fluff. I'm sure any one reading this email has the ability to write code for Google, Amazon, Oracle, or any other employer. What sets those that get hired and those who don't is how well you present that knowledge. It's not what you know, nor who you know, it's how you communicate it. -- Sent from my mobile device Envoyait de mon telephone mobil From nad at acm.org Tue Feb 16 23:30:48 2010 From: nad at acm.org (Ned Deily) Date: Tue, 16 Feb 2010 14:30:48 -0800 Subject: [Baypiggies] Resizing an image? References: <668173.93124.qm@web82302.mail.mud.yahoo.com> <20100215235014.65df1f01@dartworks.biz> <64160c71002160446n2cfa15cem93d6a6722286bfde@mail.gmail.com> <4B7AFC02.7080904@ulmcnett.com> <3fdfa9d61002161216o339cc899wd893ab340a120c18@mail.gmail.com> <4B7B0038.4090806@ulmcnett.com> Message-ID: In article , David Creemer wrote: > After struggling with PIL on Mac OS X for a while, I found the most reliable > way to develop against it is to install a "headless" Linux instance under > VirtualBox and sshfs-mount a shared file-system between systems. I edit via > Mac OS, but run under Linux. Sad but true. The added advantage for me was the > production deployment target was also Linux. Well, certainly YMMV. There are other good reasons to use a virtual Linux under OS X but just to use PIL should not be one of them. It's not *that* hard to get PIL working on OS X. FWIW, I have it installed using MacPorts on hardware ranging from a 10-year-old Macbook running 10.4 up to a current 10.6 iMac. -- Ned Deily, nad at acm.org From p at ulmcnett.com Wed Feb 17 00:52:38 2010 From: p at ulmcnett.com (Paul McNett) Date: Tue, 16 Feb 2010 15:52:38 -0800 Subject: [Baypiggies] Resizing an image? In-Reply-To: References: <668173.93124.qm@web82302.mail.mud.yahoo.com> <20100215235014.65df1f01@dartworks.biz> <64160c71002160446n2cfa15cem93d6a6722286bfde@mail.gmail.com> <4B7AFC02.7080904@ulmcnett.com> <3fdfa9d61002161216o339cc899wd893ab340a120c18@mail.gmail.com> <4B7B0038.4090806@ulmcnett.com> Message-ID: <4B7B2FC6.1090204@ulmcnett.com> On 2/16/10 1:40 PM, Ned Deily wrote: > If you are building an app for distribution to a mix of OS X systems, > you should definitely stick to building on 10.5 rather than 10.6 for the > time being if you can, especially if you are using py2app. There are > still many more potential gotchas on 10.6, primarily as a result of the > move to 64-bit as default. You can make it work and, over time, the > rough edges will get fixed but, for now, sticking with 10.5 will > undoubtedly be easier. Thanks for the MacPorts information. I'll definitely check it out. But, more importantly, thanks for the encouragement to stick with what I'm doing! :) On one hand, I'm feeling a bit conservative in my approach (still on Python 2.5.4 even, because of problems with wxPython moving forward still), but I have a reliable way to build my app for all modern Windows flavors, all current OS X flavors (10.4-10.6), and at least Ubuntu Linux. I run OS X 10.5 as my host OS, and Ubuntu/Vista as VM's for building on those platforms. All on my MacBook for maximum portability. I do all my development on this little MacBook with this little screen. Works well for me so I can relocate around the home office when the kids get too loud around my current location. I can easily install different guest VM's for Windows and Linux. Unfortunately, I can't easily install different Mac OS versions on the same machine (well, I could set them up on different partitions I guess but rebooting is a pain) so testing Mac deployments is harder/more expensive than it needs to be. I now have clients using my desktop app (Dabo-based) on Windows, Mac, and Linux, whereas just 6 months ago all my clients were on Windows. It is definitely hairy keeping up with deployment issues on all the versions of all the platforms I need to support. Deployment is the big headache of desktop apps. Paul From kbighorse at yahoo.com Wed Feb 17 01:19:23 2010 From: kbighorse at yahoo.com (Kimball Bighorse) Date: Tue, 16 Feb 2010 16:19:23 -0800 (PST) Subject: [Baypiggies] Resizing an image Message-ID: <204018.12459.qm@web82308.mail.mud.yahoo.com> Turns out it's for favicons, and apparently PIL isn't good about transparent images. Does ImageMagick/FreeImage win out in this case? We were using an alpha setting and a pythonmagickwand.wand.LANCZOS_FILTER filter, is there anything like this in PIL? Kimball From alecf at flett.org Wed Feb 17 05:19:37 2010 From: alecf at flett.org (Alec Flett) Date: Tue, 16 Feb 2010 20:19:37 -0800 Subject: [Baypiggies] Resizing an image In-Reply-To: <204018.12459.qm@web82308.mail.mud.yahoo.com> References: <204018.12459.qm@web82308.mail.mud.yahoo.com> Message-ID: My personal experience is that PIL has a very easy to use interface but falls over on a number of image formats like interlaced PNG (which are all over the web!), while GraphicsMagick (the stable fork of ImageMagick) can handle absolutely anything, but is a pain to interface with. PythonMagickWand looks pretty nice, I just wish someone would sit down and do a decent interface for the native GraphicsMagick API since at least I've found ImageMagick to be more of a pain to build/maintain! Alec On Tue, Feb 16, 2010 at 4:19 PM, Kimball Bighorse wrote: > Turns out it's for favicons, and apparently PIL isn't good about > transparent images. Does ImageMagick/FreeImage win out in this case? We > were using an alpha setting and a pythonmagickwand.wand.LANCZOS_FILTER > filter, is there anything like this in PIL? > > Kimball > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Wed Feb 17 05:36:16 2010 From: aahz at pythoncraft.com (Aahz) Date: Tue, 16 Feb 2010 20:36:16 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <2cda2fc91002161405x45a1486bja8df77ec366c59e9@mail.gmail.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> <480ED171-7523-491A-BF76-B08CD7703668@mac.com> <2cda2fc91002161405x45a1486bja8df77ec366c59e9@mail.gmail.com> Message-ID: <20100217043616.GB10044@panix.com> On Tue, Feb 16, 2010, Hasan Diwan wrote: > > Agreed with the skills improvement. I have found that the > non-technical skills are more important (especially how you come > across in an interview) and that, technical people including myself, > tend to dismiss communications skills as fluff. I'm sure any one > reading this email has the ability to write code for Google, Amazon, > Oracle, or any other employer. What sets those that get hired and > those who don't is how well you present that knowledge. It's not what > you know, nor who you know, it's how you communicate it. There's a lot of truth in that, but I have to disagree to some extent with the thesis that everyone here has the ability to write code anywhere. I certainly don't have that ability. One thing I make a point of doing when I'm looking for a job is explaining that if someone particularly wants a highly productive programmer, I am not a good fit. My background is tech support, and I still tend to solve problems from that perspective. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. :-)" From aahz at pythoncraft.com Wed Feb 17 05:41:44 2010 From: aahz at pythoncraft.com (Aahz) Date: Tue, 16 Feb 2010 20:41:44 -0800 Subject: [Baypiggies] [JOB] Python/Django Developer, San Francisco | 50-90k In-Reply-To: <1A1EA1A899AF47F99CC8F59B8DAB4382@EMACHINE> References: <1A1EA1A899AF47F99CC8F59B8DAB4382@EMACHINE> Message-ID: <20100217044144.GC10044@panix.com> On Mon, Feb 15, 2010, Beau Gould (OSS) wrote: > > Aahz brought it to my attention after I submitted the job to Python > job board for approval that there was a discussion going on here about > the salary range for this position. I'm curious why you all chat > about the low end of the range. The person that gets this job will > probably command a 90-100k salary despite the 90k cap. Perhaps a more > junior person with 6 months Django will get the job for 65k. I never > consider the required experience written in stone and you shouldn't > either. I have yet to receive one candidate thus far. There are two reasons I made a point of commenting: * It is to your advantage to avoid low-balling salary ranges -- qualified people are likely to skip over your ad with a low salary at the bottom end of the range (even if the top end is satisfactory) because they will figure that you either have no clue or are trying to play stupid games * It is to the community's advantage to have a clear idea of what we're worth and to warn people who might not be particularly career-savvy against accepting too low a salary -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. :-)" From wescpy at gmail.com Wed Feb 17 05:56:32 2010 From: wescpy at gmail.com (wesley chun) Date: Tue, 16 Feb 2010 20:56:32 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <480ED171-7523-491A-BF76-B08CD7703668@mac.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> <480ED171-7523-491A-BF76-B08CD7703668@mac.com> Message-ID: <78b3a9581002162056s6b3a33e6p547dea79ea33ed6f@mail.gmail.com> i'm late too. however, i don't have much to add... everyone else has already said it. i wonder if this salary range is meant to target H1B individuals, where there is some hidden cost of sponsorship and being able to stay here in the US. -wesley On Tue, Feb 16, 2010 at 12:49 PM, Grace Law wrote: > Sorry for the late entry to this thread. > IMHO, $50K is exploitation, even for an average developer, but many > companies do that and think it is OK. -- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - "Python Web Development with Django", Addison Wesley, (c) 2009 http://withdjango.com wesley.j.chun :: wescpy-at-gmail.com python training and technical consulting cyberweb.consulting : silicon valley, ca http://cyberwebconsulting.com From brent.tubbs at gmail.com Wed Feb 17 06:45:22 2010 From: brent.tubbs at gmail.com (Brent Tubbs) Date: Tue, 16 Feb 2010 21:45:22 -0800 Subject: [Baypiggies] Salary ranges In-Reply-To: <78b3a9581002162056s6b3a33e6p547dea79ea33ed6f@mail.gmail.com> References: <20100215154007.GA2545@panix.com> <20100215165636.GA27630@panix.com> <5df1e9b01002151012t7a73d705q19deb87292df416e@mail.gmail.com> <480ED171-7523-491A-BF76-B08CD7703668@mac.com> <78b3a9581002162056s6b3a33e6p547dea79ea33ed6f@mail.gmail.com> Message-ID: <769bb4301002162145l336ea4d6lfa35f54fde7c1ac0@mail.gmail.com> Though note that an employer that did that would be violating immigration law and flouting the Labor Condition Statement agreed to as part of the H1B application: (1) Wages: The employer attests that H-1B, H-1B1 or E-3 foreign workers will be paid wages which are at least the higher of the actual wage level paid by the employer to all other individuals with similar experience and qualifications for the specific employment in question or the prevailing wage level for occupational classification in the area of intended employment. See http://www.foreignlaborcert.doleta.gov/preh1bform.cfm and http://www.foreignlaborcert.doleta.gov/pdf/ETA_Form_9035CP_2009_Revised.pdf. On Tue, Feb 16, 2010 at 8:56 PM, wesley chun wrote: > i'm late too. however, i don't have much to add... everyone else has > already said it. i wonder if this salary range is meant to target H1B > individuals, where there is some hidden cost of sponsorship and being > able to stay here in the US. > > -wesley > > > On Tue, Feb 16, 2010 at 12:49 PM, Grace Law wrote: > > Sorry for the late entry to this thread. > > IMHO, $50K is exploitation, even for an average developer, but many > > companies do that and think it is OK. > > > -- > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > "Python Web Development with Django", Addison Wesley, (c) 2009 > http://withdjango.com > > wesley.j.chun :: wescpy-at-gmail.com > python training and technical consulting > cyberweb.consulting : silicon valley, ca > http://cyberwebconsulting.com > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at zachary.com Wed Feb 17 10:14:06 2010 From: david at zachary.com (David Creemer) Date: Wed, 17 Feb 2010 01:14:06 -0800 Subject: [Baypiggies] Resizing an image In-Reply-To: References: <204018.12459.qm@web82308.mail.mud.yahoo.com> Message-ID: <9DD0E652-EFC0-4094-B099-2E75D2C61246@zachary.com> Be sure to see PIL 1.1.7: http://hg.effbot.org/pil-117/ I'm using PIL 1.1.7 as part of a caching image proxy that needs to perform a variety of transformations of remote images. With 1.1.7 and some additions, I can now handle all image formats that matter to me. It handles interlaced PNGs, as well as a large variety of favicons (using this code: http://code.google.com/p/casadebender/wiki/Win32IconImagePlugin). The only image format not easily handled that I needed was SVG. For that, I use librsvg2 to render SVG to PNG. I originally built my project on ImageMagick / MagickWand, but eventually switched to PIL 1.1.7 due to it's cleaner interface, better docs, and more flexibility. -- David On Feb 16, 2010, at 8:19 PM, Alec Flett wrote: > My personal experience is that PIL has a very easy to use interface but falls over on a number of image formats like interlaced PNG (which are all over the web!), while GraphicsMagick (the stable fork of ImageMagick) can handle absolutely anything, but is a pain to interface with. > > PythonMagickWand looks pretty nice, I just wish someone would sit down and do a decent interface for the native GraphicsMagick API since at least I've found ImageMagick to be more of a pain to build/maintain! > > Alec > > On Tue, Feb 16, 2010 at 4:19 PM, Kimball Bighorse wrote: > Turns out it's for favicons, and apparently PIL isn't good about transparent images. Does ImageMagick/FreeImage win out in this case? We were using an alpha setting and a pythonmagickwand.wand.LANCZOS_FILTER filter, is there anything like this in PIL? > > Kimball > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies -------------- next part -------------- An HTML attachment was scrubbed... URL: From roderick at sanfransystems.com Wed Feb 17 17:33:45 2010 From: roderick at sanfransystems.com (Roderick Llewellyn) Date: Wed, 17 Feb 2010 08:33:45 -0800 Subject: [Baypiggies] Salary Ranges References: Message-ID: I applied to one job (Calypso Technology, in the financial communications space), for which I had very good qualifications IMHO. They had a typical automated form, which required about 30-40 minutes of cutting and pasting from my resume, as it had separate entry fields for each job, each employer, dates, etc (rather than just pasting my entire resume), and also asked for skills ratings for various technologies. They had the requirement of filling in a numerical desired salary (you could not proceed without doing so). I entered $100,000, not believing in low-balling either. At 9:34 in the morning, moments after submitting the form, I got an automated reply acknowledging my submssion. At 9:37 -- three minutes later -- I got a rejection notice from the same mailing address. Obviously no human looked at my qualifications. I was rejected by a computer program that said "IF desired_salary > tiny_amount_we_want_to_pay THEN Reject(candidate)". The good news is it took only 3 minutes instead of waiting around for weeks and never hearing back from the employer, which is more typical American practice. Should I have "low-balled" the salary requirement? Tried to out-guess their program? Just for fun, here's another horror story. I was contacted by Facebook (thru LinkedIn), by an internal recruiter, someone named Yancy Rivera. (I don't mind naming names!). After some discussion, it sounded like I qualified for the position, so he told me the next step was to go on their engineering page and solve one of two "puzzles" held therein. These puzzles are computer programs you must write in any of several languages. I chose Python (doh!) In case anybody is familiar with these, I picked the "User Bin Crash" problem. It's a typical NP-complete type problem.You prepare your program and submit it to its "Puzzlebot" which every 4 hours runs your program and presents it with various test cases. If your program doesn't pass, the bot gives you a rejection email. You are told nothing else; you are not given test data, you are not given exceptions, you are not even told that your program took too long to run. Is this a realistic debugging scenario? Now my mistake was not doing what every modern programmer would have done: cheated. I should have just downloaded a solution off the web. Maybe that was the real test. After all, few companies actually want you to do any real programming these days, they want you to download free software and tweak its XML configuration parameters. Anyway, after many submissions, I was still unable to pass their bot, tho my program passed every test case I threw at it. So I finally broke down and downloaded a Python solution off the web that supposedly passed Puzzlebot. I analyzed this solution and discovered a flaw! If your test case contained a large prime number, it would blow memory and could take a very long time. Sure enough, when I gave it the largest prime < 1 billion (there's a table on the web, of course), the "working" solution failed by blowing memory but it passed my solution handily. However, for many cases, the "working" solution was somewhat faster than mine. Almost certainly that was why my solution did not pass Puzzlebot. So I wrote to the recruiter explaining this, saying that "this is the problem with using a machine to judge a man". With my mind, I was able to analyze a supposedly working solution, one that had passed Puzzlebot, discovering a major flaw. Letting Puzzlebot decide who gets hired would not have brought in the most analytical mind, merely one good at fooling a machine. He promised to contact Engineering and, at long last, actually run my resume past a technical person. About 15 minutes later I got a "thanks but no thanks" from them. After 20 hours of programming, they spent a quarter of an hour of their technical people's time on me. Maybe much less. Needless to say, I won't be using Facebook anymore! Have fun, Rod Llewellyn From wescpy at gmail.com Wed Feb 17 17:47:40 2010 From: wescpy at gmail.com (wesley chun) Date: Wed, 17 Feb 2010 08:47:40 -0800 Subject: [Baypiggies] Salary Ranges In-Reply-To: References: Message-ID: <78b3a9581002170847i21754847jc4ee7b42700959a2@mail.gmail.com> i'd write a script to keep on submitting to the machine until i don't get rejected in order to see what value is stored in tiny_amount_we_want_to_pay. :-) On Wed, Feb 17, 2010 at 8:33 AM, Roderick Llewellyn wrote: > I applied to one job (Calypso Technology, in the financial communications > space), for which I had very good qualifications IMHO. They had a typical > automated form, which required about 30-40 minutes of cutting and pasting > from my resume, as it had separate entry fields for each job, each employer, > dates, etc (rather than just pasting my entire resume), and also asked for > skills ratings for various technologies. They had the requirement of filling > in a numerical desired salary (you could not proceed without doing so). I > entered $100,000, not believing in low-balling either. At 9:34 in the > morning, moments after submitting the form, I got an automated reply > acknowledging my submssion. At 9:37 -- three minutes later -- I got a > rejection notice from the same mailing address. Obviously no human looked at > my qualifications. I was rejected by a computer program that said "IF > desired_salary > tiny_amount_we_want_to_pay THEN Reject(candidate)". ?The > good news is it took only 3 minutes instead of waiting around for weeks and > never hearing back from the employer, which is more typical American > practice. Should I have "low-balled" the salary requirement? Tried to > out-guess their program? -- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - "Python Web Development with Django", Addison Wesley, (c) 2009 http://withdjango.com wesley.j.chun :: wescpy-at-gmail.com python training and technical consulting cyberweb.consulting : silicon valley, ca http://cyberwebconsulting.com From luca.pellicoro at gmail.com Wed Feb 17 18:35:12 2010 From: luca.pellicoro at gmail.com (Luca Pellicoro) Date: Wed, 17 Feb 2010 09:35:12 -0800 Subject: [Baypiggies] Salary Ranges In-Reply-To: <78b3a9581002170847i21754847jc4ee7b42700959a2@mail.gmail.com> References: <78b3a9581002170847i21754847jc4ee7b42700959a2@mail.gmail.com> Message-ID: On Wednesday, February 17, 2010, wesley chun wrote: > i'd write a script to keep on submitting to the machine until i don't > get rejected in order to see what value is stored in > tiny_amount_we_want_to_pay. :-) I'd use selenium so it looks like a real browser from their end. Putting in garbage data until I got the right number. Maybe even masking my ip. > > On Wed, Feb 17, 2010 at 8:33 AM, Roderick Llewellyn > wrote: >> I applied to one job (Calypso Technology, in the financial communications >> space), for which I had very good qualifications IMHO. They had a typical >> automated form, which required about 30-40 minutes of cutting and pasting >> from my resume, as it had separate entry fields for each job, each employer, >> dates, etc (rather than just pasting my entire resume), and also asked for >> skills ratings for various technologies. They had the requirement of filling >> in a numerical desired salary (you could not proceed without doing so). I >> entered $100,000, not believing in low-balling either. At 9:34 in the >> morning, moments after submitting the form, I got an automated reply >> acknowledging my submssion. At 9:37 -- three minutes later -- I got a >> rejection notice from the same mailing address. Obviously no human looked at >> my qualifications. I was rejected by a computer program that said "IF >> desired_salary > tiny_amount_we_want_to_pay THEN Reject(candidate)". ?The >> good news is it took only 3 minutes instead of waiting around for weeks and >> never hearing back from the employer, which is more typical American >> practice. Should I have "low-balled" the salary requirement? Tried to >> out-guess their program? > > > -- > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > "Python Web Development with Django", Addison Wesley, (c) 2009 > ? ?http://withdjango.com > > wesley.j.chun :: wescpy-at-gmail.com > python training and technical consulting > cyberweb.consulting : silicon valley, ca > http://cyberwebconsulting.com > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From rich at noir.com Wed Feb 17 18:58:31 2010 From: rich at noir.com (K. Richard Pixley) Date: Wed, 17 Feb 2010 09:58:31 -0800 Subject: [Baypiggies] Salary Ranges In-Reply-To: References: <78b3a9581002170847i21754847jc4ee7b42700959a2@mail.gmail.com> Message-ID: <4B7C2E47.1060305@noir.com> Luca Pellicoro wrote: > On Wednesday, February 17, 2010, wesley chun wrote: > >> i'd write a script to keep on submitting to the machine until i don't >> get rejected in order to see what value is stored in >> tiny_amount_we_want_to_pay. :-) >> > I'd use selenium so it looks like a real browser from their end. > Putting in garbage data until I got the right number. Maybe even > masking my ip. Lol. Bah! I say let'em know who has the ingenuity, skllz, chutzpah to sort them out. :). (Like they'd ever notice.) --rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From puruiw at yahoo.com Wed Feb 17 19:53:28 2010 From: puruiw at yahoo.com (Purui Wang) Date: Wed, 17 Feb 2010 10:53:28 -0800 (PST) Subject: [Baypiggies] Salary Ranges In-Reply-To: References: <78b3a9581002170847i21754847jc4ee7b42700959a2@mail.gmail.com> Message-ID: <7609.65632.qm@web45002.mail.sp1.yahoo.com> ________________________________ From: Luca Pellicoro To: wesley chun Cc: baypiggies at python.org; Roderick Llewellyn Sent: Wed, February 17, 2010 9:35:12 AM Subject: Re: [Baypiggies] Salary Ranges On Wednesday, February 17, 2010, wesley chun wrote: > i'd write a script to keep on submitting to the machine until i don't > get rejected in order to see what value is stored in > tiny_amount_we_want_to_pay. :-) I'd use selenium so it looks like a real browser from their end. Putting in garbage data until I got the right number. Maybe even masking my ip. A simple "Are you human?" box can void all your efforts. - Purui -------------- next part -------------- An HTML attachment was scrubbed... URL: From dpb at donbennett.org Wed Feb 17 21:01:45 2010 From: dpb at donbennett.org (Don Bennett) Date: Wed, 17 Feb 2010 12:01:45 -0800 Subject: [Baypiggies] Salary Ranges In-Reply-To: References: Message-ID: <4e67cc901002171201s1d56a377q73213882a8949329@mail.gmail.com> After failing to get a Python solution to pass the Puzzlebot, and switched to C++ and received a 'pass' with the same algorithm; I haven't gone back to any of my other solutions to see if I can get a passing solution in Python... Don On Wed, Feb 17, 2010 at 8:33 AM, Roderick Llewellyn wrote: > I applied to one job (Calypso Technology, in the financial communications > space), for which I had very good qualifications IMHO. They had a typical > automated form, which required about 30-40 minutes of cutting and pasting > from my resume, as it had separate entry fields for each job, each employer, > dates, etc (rather than just pasting my entire resume), and also asked for > skills ratings for various technologies. They had the requirement of filling > in a numerical desired salary (you could not proceed without doing so). I > entered $100,000, not believing in low-balling either. At 9:34 in the > morning, moments after submitting the form, I got an automated reply > acknowledging my submssion. At 9:37 -- three minutes later -- I got a > rejection notice from the same mailing address. Obviously no human looked at > my qualifications. I was rejected by a computer program that said "IF > desired_salary > tiny_amount_we_want_to_pay THEN Reject(candidate)". ?The > good news is it took only 3 minutes instead of waiting around for weeks and > never hearing back from the employer, which is more typical American > practice. Should I have "low-balled" the salary requirement? Tried to > out-guess their program? > > Just for fun, here's another horror story. I was contacted by Facebook (thru > LinkedIn), by an internal recruiter, someone named Yancy Rivera. (I don't > mind naming names!). After some discussion, it sounded like I qualified for > the position, so he told me the next step was to go on their engineering > page and solve one of two "puzzles" held therein. These puzzles are computer > programs you must write in any of several languages. I chose Python (doh!) > In case anybody is familiar with these, I picked the "User Bin Crash" > problem. It's a typical NP-complete type problem.You prepare your program > and submit it to its "Puzzlebot" which every 4 hours runs your program and > presents it with various test cases. If your program doesn't pass, the bot > gives you a rejection email. You are told nothing else; you are not given > test data, you are not given exceptions, you are not even told that your > program took too long to run. Is this a realistic debugging scenario? > > Now my mistake was not doing what every modern programmer would have done: > cheated. I should have just downloaded a solution off the web. Maybe that > was the real test. After all, few companies actually want you to do any real > programming these days, they want you to download free software and tweak > its XML configuration parameters. Anyway, after many submissions, I was > still unable to pass their bot, tho my program passed every test case I > threw at it. So I finally broke down and downloaded a Python solution off > the web that supposedly passed Puzzlebot. I analyzed this solution and > discovered a flaw! If your test case contained a large prime number, it > would blow memory and could take a very long time. Sure enough, when I gave > it the largest prime < 1 billion (there's a table on the web, of course), > the "working" solution failed by blowing memory but it passed my solution > handily. However, for many cases, the "working" solution was somewhat faster > than mine. Almost certainly that was why my solution did not pass Puzzlebot. > > So I wrote to the recruiter explaining this, saying that "this is the > problem with using a machine to judge a man". With my mind, I was able to > analyze a supposedly working solution, one that had passed Puzzlebot, > discovering a major flaw. Letting Puzzlebot decide who gets hired would not > have brought in the most analytical mind, merely one good at fooling a > machine. He promised to contact Engineering and, at long last, actually run > my resume past a technical person. About 15 minutes later I got a "thanks > but no thanks" from them. After 20 hours of programming, they spent a > quarter of an hour of their technical people's time on me. Maybe much less. > Needless to say, I won't be using Facebook anymore! > > Have fun, > > Rod Llewellyn > > > > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From andrew at atoulou.se Wed Feb 17 21:24:19 2010 From: andrew at atoulou.se (Andrew Akira Toulouse) Date: Wed, 17 Feb 2010 12:24:19 -0800 Subject: [Baypiggies] Salary Ranges In-Reply-To: <4e67cc901002171201s1d56a377q73213882a8949329@mail.gmail.com> References: <4e67cc901002171201s1d56a377q73213882a8949329@mail.gmail.com> Message-ID: More details, please, so I can file a bug report with facebook (I sat near a guy on the puzzle team when I interned there, and could inquire) - they're really an awesome company, but their best recruiter left for a younger, shinier company, and their hiring practices haven't really scaled too well over the past year as far as I can tell. On Wed, Feb 17, 2010 at 12:01 PM, Don Bennett wrote: > After failing to get a Python solution to pass the Puzzlebot, and > switched to C++ and received a 'pass' with the same algorithm; I > haven't gone back to any of my other solutions to see if I can get a > passing solution in Python... > > Don > > On Wed, Feb 17, 2010 at 8:33 AM, Roderick Llewellyn > wrote: >> I applied to one job (Calypso Technology, in the financial communications >> space), for which I had very good qualifications IMHO. They had a typical >> automated form, which required about 30-40 minutes of cutting and pasting >> from my resume, as it had separate entry fields for each job, each employer, >> dates, etc (rather than just pasting my entire resume), and also asked for >> skills ratings for various technologies. They had the requirement of filling >> in a numerical desired salary (you could not proceed without doing so). I >> entered $100,000, not believing in low-balling either. At 9:34 in the >> morning, moments after submitting the form, I got an automated reply >> acknowledging my submssion. At 9:37 -- three minutes later -- I got a >> rejection notice from the same mailing address. Obviously no human looked at >> my qualifications. I was rejected by a computer program that said "IF >> desired_salary > tiny_amount_we_want_to_pay THEN Reject(candidate)". ?The >> good news is it took only 3 minutes instead of waiting around for weeks and >> never hearing back from the employer, which is more typical American >> practice. Should I have "low-balled" the salary requirement? Tried to >> out-guess their program? >> >> Just for fun, here's another horror story. I was contacted by Facebook (thru >> LinkedIn), by an internal recruiter, someone named Yancy Rivera. (I don't >> mind naming names!). After some discussion, it sounded like I qualified for >> the position, so he told me the next step was to go on their engineering >> page and solve one of two "puzzles" held therein. These puzzles are computer >> programs you must write in any of several languages. I chose Python (doh!) >> In case anybody is familiar with these, I picked the "User Bin Crash" >> problem. It's a typical NP-complete type problem.You prepare your program >> and submit it to its "Puzzlebot" which every 4 hours runs your program and >> presents it with various test cases. If your program doesn't pass, the bot >> gives you a rejection email. You are told nothing else; you are not given >> test data, you are not given exceptions, you are not even told that your >> program took too long to run. Is this a realistic debugging scenario? >> >> Now my mistake was not doing what every modern programmer would have done: >> cheated. I should have just downloaded a solution off the web. Maybe that >> was the real test. After all, few companies actually want you to do any real >> programming these days, they want you to download free software and tweak >> its XML configuration parameters. Anyway, after many submissions, I was >> still unable to pass their bot, tho my program passed every test case I >> threw at it. So I finally broke down and downloaded a Python solution off >> the web that supposedly passed Puzzlebot. I analyzed this solution and >> discovered a flaw! If your test case contained a large prime number, it >> would blow memory and could take a very long time. Sure enough, when I gave >> it the largest prime < 1 billion (there's a table on the web, of course), >> the "working" solution failed by blowing memory but it passed my solution >> handily. However, for many cases, the "working" solution was somewhat faster >> than mine. Almost certainly that was why my solution did not pass Puzzlebot. >> >> So I wrote to the recruiter explaining this, saying that "this is the >> problem with using a machine to judge a man". With my mind, I was able to >> analyze a supposedly working solution, one that had passed Puzzlebot, >> discovering a major flaw. Letting Puzzlebot decide who gets hired would not >> have brought in the most analytical mind, merely one good at fooling a >> machine. He promised to contact Engineering and, at long last, actually run >> my resume past a technical person. About 15 minutes later I got a "thanks >> but no thanks" from them. After 20 hours of programming, they spent a >> quarter of an hour of their technical people's time on me. Maybe much less. >> Needless to say, I won't be using Facebook anymore! >> >> Have fun, >> >> Rod Llewellyn >> >> >> >> >> >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> http://mail.python.org/mailman/listinfo/baypiggies >> > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From jsnitow at gmail.com Wed Feb 17 22:34:15 2010 From: jsnitow at gmail.com (Julian Snitow) Date: Wed, 17 Feb 2010 13:34:15 -0800 Subject: [Baypiggies] Salary Ranges In-Reply-To: <7609.65632.qm@web45002.mail.sp1.yahoo.com> References: <78b3a9581002170847i21754847jc4ee7b42700959a2@mail.gmail.com> <7609.65632.qm@web45002.mail.sp1.yahoo.com> Message-ID: <54075e091002171334k519c0be3i708cf669089d570e@mail.gmail.com> On Wed, Feb 17, 2010 at 10:53 AM, Purui Wang wrote: > > A simple "Are you human?" box can void all your?efforts. As beings of pure logic, robots are incapable of deceit. ;-) From andrew at atoulou.se Wed Feb 17 23:06:14 2010 From: andrew at atoulou.se (Andrew Akira Toulouse) Date: Wed, 17 Feb 2010 14:06:14 -0800 Subject: [Baypiggies] Programming Puzzles (discussion forked from Salary Ranges) Message-ID: Getting rid of puzzles at Facebook is a suggestion they are unlikely to agree with, as it has worked well for them. I know firsthand that the puzzles have helped the recruiters find several highly skilled candidates who have become productive employees there. They are not only looking for the best communicators, but excellent problem-solvers. When I said their hiring practices haven't scaled, I meant that there are more false negatives than false positives - that is, they pass over good hires but very rarely make poor hiring decisions, and have fired very few people for incompetence, which they are not afraid to do. I know a lot of people at Facebook and the only person I can think of who started to approach the lone ranger stereotype was someone who was working on an extremely hard optimization problem that required heavy engineering experience and rare algorithmic talent. That said, I asked if you have any bug reports for the puzzlebot, as I can pass them on. I'm not a representative of Facebook, and my internship ended a few months ago. but I still have connections. --Andy On Wed, Feb 17, 2010 at 12:53 PM, Fred C wrote: > A good start will be to stop using these puzzle. A good 1/1 with people is the best to find talents. The only purpose of a puzzle is to shield the hiring team from management, when someone incompetent is hired. It can then say: "I don't understand, he successfully passed all our toughest tests". At the end a good team is a collection of individual who can interact and work together, with complementary skills. Not a bunch of puzzle solvers. I don't count the numerous times I had to work with someone very skilled at solving puzzle but incapable of working in a team. > > > >> >> On Wed, Feb 17, 2010 at 12:01 PM, Don Bennett wrote: >>> After failing to get a Python solution to pass the Puzzlebot, and >>> switched to C++ and received a 'pass' with the same algorithm; I >>> haven't gone back to any of my other solutions to see if I can get a >>> passing solution in Python... >>> >>> Don >>> >>> On Wed, Feb 17, 2010 at 8:33 AM, Roderick Llewellyn >>> wrote: >>>> I applied to one job (Calypso Technology, in the financial communications >>>> space), for which I had very good qualifications IMHO. They had a typical >>>> automated form, which required about 30-40 minutes of cutting and pasting >>>> from my resume, as it had separate entry fields for each job, each employer, >>>> dates, etc (rather than just pasting my entire resume), and also asked for >>>> skills ratings for various technologies. They had the requirement of filling >>>> in a numerical desired salary (you could not proceed without doing so). I >>>> entered $100,000, not believing in low-balling either. At 9:34 in the >>>> morning, moments after submitting the form, I got an automated reply >>>> acknowledging my submssion. At 9:37 -- three minutes later -- I got a >>>> rejection notice from the same mailing address. Obviously no human looked at >>>> my qualifications. I was rejected by a computer program that said "IF >>>> desired_salary > tiny_amount_we_want_to_pay THEN Reject(candidate)". ?The >>>> good news is it took only 3 minutes instead of waiting around for weeks and >>>> never hearing back from the employer, which is more typical American >>>> practice. Should I have "low-balled" the salary requirement? Tried to >>>> out-guess their program? >>>> >>>> Just for fun, here's another horror story. I was contacted by Facebook (thru >>>> LinkedIn), by an internal recruiter, someone named Yancy Rivera. (I don't >>>> mind naming names!). After some discussion, it sounded like I qualified for >>>> the position, so he told me the next step was to go on their engineering >>>> page and solve one of two "puzzles" held therein. These puzzles are computer >>>> programs you must write in any of several languages. I chose Python (doh!) >>>> In case anybody is familiar with these, I picked the "User Bin Crash" >>>> problem. It's a typical NP-complete type problem.You prepare your program >>>> and submit it to its "Puzzlebot" which every 4 hours runs your program and >>>> presents it with various test cases. If your program doesn't pass, the bot >>>> gives you a rejection email. You are told nothing else; you are not given >>>> test data, you are not given exceptions, you are not even told that your >>>> program took too long to run. Is this a realistic debugging scenario? >>>> >>>> Now my mistake was not doing what every modern programmer would have done: >>>> cheated. I should have just downloaded a solution off the web. Maybe that >>>> was the real test. After all, few companies actually want you to do any real >>>> programming these days, they want you to download free software and tweak >>>> its XML configuration parameters. Anyway, after many submissions, I was >>>> still unable to pass their bot, tho my program passed every test case I >>>> threw at it. So I finally broke down and downloaded a Python solution off >>>> the web that supposedly passed Puzzlebot. I analyzed this solution and >>>> discovered a flaw! If your test case contained a large prime number, it >>>> would blow memory and could take a very long time. Sure enough, when I gave >>>> it the largest prime < 1 billion (there's a table on the web, of course), >>>> the "working" solution failed by blowing memory but it passed my solution >>>> handily. However, for many cases, the "working" solution was somewhat faster >>>> than mine. Almost certainly that was why my solution did not pass Puzzlebot. >>>> >>>> So I wrote to the recruiter explaining this, saying that "this is the >>>> problem with using a machine to judge a man". With my mind, I was able to >>>> analyze a supposedly working solution, one that had passed Puzzlebot, >>>> discovering a major flaw. Letting Puzzlebot decide who gets hired would not >>>> have brought in the most analytical mind, merely one good at fooling a >>>> machine. He promised to contact Engineering and, at long last, actually run >>>> my resume past a technical person. About 15 minutes later I got a "thanks >>>> but no thanks" from them. After 20 hours of programming, they spent a >>>> quarter of an hour of their technical people's time on me. Maybe much less. >>>> Needless to say, I won't be using Facebook anymore! >>>> >>>> Have fun, >>>> >>>> Rod Llewellyn >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Baypiggies mailing list >>>> Baypiggies at python.org >>>> To change your subscription options or unsubscribe: >>>> http://mail.python.org/mailman/listinfo/baypiggies >>>> >>> _______________________________________________ >>> Baypiggies mailing list >>> Baypiggies at python.org >>> To change your subscription options or unsubscribe: >>> http://mail.python.org/mailman/listinfo/baypiggies >>> >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> http://mail.python.org/mailman/listinfo/baypiggies > > > From jeffrey.fischer at gmail.com Thu Feb 18 05:04:40 2010 From: jeffrey.fischer at gmail.com (Jeff Fischer) Date: Wed, 17 Feb 2010 20:04:40 -0800 Subject: [Baypiggies] =?windows-1252?q?=5BJOB=5D_Cloud_Computing_Startup_?= =?windows-1252?q?=96_Python_Developer_=28Part-time_Contract=2C_Sun?= =?windows-1252?q?nyvale=29?= Message-ID: <2c95bbb01002172004u782d917dg14909619d6b4cf59@mail.gmail.com> genForma is a stealth-mode startup pioneering a new way to run open source applications in the cloud. We have seed funding from the National Science Foundation are looking for a part-time developer with a passion for open source and learning new technologies. This position involves developing Python scripts for automating system management tasks in a cloud environment. There may also be some work on modifying/improving open source applications written in Python, Java, PHP, etc. Experience with HTML/CSS/JavaScript is also helpful. The contract is for 2.5 months at 20 - 25 hours per week, with the possibility of extending it past the initial time-frame. If you like a dynamic environment, lots of autonomy, working on open source and cloud computing, then this is the job for you! Requirements - Must have a B.S. degree in Computer Science / Computer Engineering - Must have the legal right to work in the U.S. - Must be located in the San Francisco bay area and available for weekly in-person meetings in Sunnyvale. - Principals only. No recruiters or outsoucing firms. Desired skills - Python scripting experience, particularly for system management tasks - Basic knowledge of Linux system administration - Excellent verbal and written communications skills - HTML/CSS/JavaScript experience a plus - Experience with cloud computing technologies (e.g. Libcloud, Virtual Private Servers, Google App engine, Amazon Web Services) - Experience with popular open source applications (e.g. Blogs, Project management, Wikis) - Knowledge of Java, PHP, Perl, and/or OCaml a plus To apply for this position, please send a cover letter and resume to jobs at genforma.com. From dalke at dalkescientific.com Thu Feb 18 14:23:07 2010 From: dalke at dalkescientific.com (Andrew Dalke) Date: Thu, 18 Feb 2010 08:23:07 -0500 Subject: [Baypiggies] Programming Puzzles (discussion forked from Salary Ranges) In-Reply-To: References: Message-ID: <862CE7C8-138D-4B8C-835B-4BA935EA68FA@dalkescientific.com> On Feb 17, 2010, at 5:06 PM, Andrew Akira Toulouse wrote: > Getting rid of puzzles at Facebook is a suggestion they are unlikely > to agree with, as it has worked well for them. ... They are not > only looking for the best communicators, but excellent > problem-solvers. And the correlation between puzzle-solvers of the sort which can be done during an interview and problem-solvers of the sort which might take days of work is ... ? I ask this because the major reason I can do those common interview puzzles is because I've read most of them in case they were asked of me some day. While I've spent a month developing a solution to find the maximum common subgraph in a set of N graphs. (Rather, of substructures in a group of molecules.) > the only person I can think of who started to approach the > lone ranger stereotype was someone who was working on an > extremely hard optimization problem that required heavy > engineering experience and rare algorithmic talent. Hmm.... And perhaps the space I work in has algorithmically harder problems than Facebook. Andrew dalke at dalkescientific.com From niallo at unworkable.org Fri Feb 19 05:42:47 2010 From: niallo at unworkable.org (Niall O'Higgins) Date: Thu, 18 Feb 2010 20:42:47 -0800 Subject: [Baypiggies] Py Web SF #8: Bram Cohen "New Python Logging Framework" - Feb 23rd 6pm @ SFPL Message-ID: <20100219044247.GK7032@unworkable.org> Hi folks, PyWebSF is a Python meet-up with a strong focus on Web technology. From frameworks like WSGI/Pylons/TurboGears/Django to libraries like httplib2 to using emerging Web technologies like Amazon's AWS and Freebase - its all covered. The emphasis is on practical, hands-on lectures and discussion. Meetings start with a 45-60 minute presentation and end with informal discussion. Hackathon-style collaboration and project demos are encouraged. Who/What -------- * Bram Cohen - "A New Python Logging Framework" http://www.pywebsf.org/2010/02/09/bram-cohen-new-python-logging-framework/ When ---- 6PM, Tuesday 23 February 2010. Please try to arrive on time to avoid disappointment. We have space for around 10-20 people. Where ----- Stong conference room, 1st floor, SF Main Public Library. Map: http://tinyurl.com/pywebsfmap The library is easily accessible via both BART and Muni at the Civic Center station. The library closes at 8pm so we will continue the discussion over food/drinks at Frjtz Fries [http://www.frjtzfries.com]. More info --------- Subscribe to our Google Calendar at http://tinyurl.com/pywebcal Slides, links, and more at http://pywebsf.org/ Thanks! -- Niall O'Higgins PyWebSF http://pywebsf.org http://niallohiggins.com http://twitter.com/niallohiggins From beau at open-source-staffing.com Fri Feb 19 09:02:27 2010 From: beau at open-source-staffing.com (Beau Gould (OSS)) Date: Fri, 19 Feb 2010 03:02:27 -0500 Subject: [Baypiggies] [JOB] Python/Django Developer, San Francisco, CA | 70-90k **UPDATED** Message-ID: <01C023B10F794190B38C52FA274B022E@EMACHINE> This is a full time, on-site, salaried position located in San Francisco, CA (94107) paying $70,000 to $90,000 (dep on exp) + benefits. US Citizens, Green Card holders, EAD or CAN only please. Local candidates only. No telecommuting. Thank you. My client is a geographic data and services company. They collect map data and distribute it through a hosted geoservices platform. Their flagship product is used by all major portals, yellow pages sites and a leading collection of mobile and social applications. You are a San Francisco-area based web developer who has solid experience writing software for the web. You can develop web pages with HTML, JavaScript, and CSS, and are also comfortable writing code which queries databases, accesses web services, and processes data from the command line. Required experience: - proven web front-end experience with HTML/CSS - 2+ years experience with Python Other desirable experience: - Django experience a HUGE plus! - Ruby and Java - Web mapping, GIS, cartography - Amazon Web Services - Postgres/PostGIS, GEOS/GDAL/OGR - Flex/Flash - Data Warehousing - RESTful API design - Linux administration To be considered, please submit your resume and salary requirements to beau[AT]open-source-staffing.com Thank you, Beau J. Gould ------------------ Open Source Staffing http://www.open-source-staffing.com http://www.facebook.com/beau.gould beau[AT]open-source-staffing.com From andrew at atoulou.se Fri Feb 19 09:34:44 2010 From: andrew at atoulou.se (Andrew Akira Toulouse) Date: Fri, 19 Feb 2010 00:34:44 -0800 Subject: [Baypiggies] Programming Puzzles (discussion forked from Salary Ranges) In-Reply-To: <862CE7C8-138D-4B8C-835B-4BA935EA68FA@dalkescientific.com> References: <862CE7C8-138D-4B8C-835B-4BA935EA68FA@dalkescientific.com> Message-ID: I think you misunderstand the puzzles I'm talking about: http://www.facebook.com/careers/puzzles.php These are puzzles you work on at your own pace, with a variety of language choices. You can certainly spend a month coding a super-awesome solution. In fact, it's often noticed by the engineers making the puzzles and thus by HR if you submit enough well-executed solutions. --Andy On Thu, Feb 18, 2010 at 5:23 AM, Andrew Dalke wrote: > On Feb 17, 2010, at 5:06 PM, Andrew Akira Toulouse wrote: >> Getting rid of puzzles at Facebook is a suggestion they are unlikely >> to agree with, as it has worked well for them. ?... They are not >> only looking for the best communicators, but excellent >> problem-solvers. > > And the correlation between puzzle-solvers of the sort > which can be done during an interview and problem-solvers > of the sort which might take days of work is ... ? > > I ask this because the major reason I can do those common > interview puzzles is because I've read most of them in > case they were asked of me some day. > > While I've spent a month developing a solution to find the > maximum common subgraph in a set of N graphs. (Rather, > of substructures in a group of molecules.) > >> the only person I can think of who started to approach the >> lone ranger stereotype was someone who was working on an >> extremely hard optimization problem that required heavy >> engineering experience and rare algorithmic talent. > > Hmm.... And perhaps the space I work in has algorithmically > harder problems than Facebook. > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Andrew > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?dalke at dalkescientific.com > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From dalke at dalkescientific.com Fri Feb 19 14:34:19 2010 From: dalke at dalkescientific.com (Andrew Dalke) Date: Fri, 19 Feb 2010 08:34:19 -0500 Subject: [Baypiggies] Programming Puzzles (discussion forked from Salary Ranges) In-Reply-To: References: <862CE7C8-138D-4B8C-835B-4BA935EA68FA@dalkescientific.com> Message-ID: On Feb 19, 2010, at 3:34 AM, Andrew Akira Toulouse wrote: > I think you misunderstand the puzzles I'm talking about: > http://www.facebook.com/careers/puzzles.php Indeed I did. I didn't see that in the thread discussion and a search for "Facebook puzzle" found rather a lot of other things. Thanks for the correction! Andrew dalke at dalkescientific.com From bret.clement at gmail.com Fri Feb 19 23:45:00 2010 From: bret.clement at gmail.com (Bret Clement) Date: Fri, 19 Feb 2010 15:45:00 -0700 Subject: [Baypiggies] New Python Code Recipe site by ActiveState and O'Reilly Message-ID: <94abcabc1002191445s6b311d0dj2dd4ee839bc0c365@mail.gmail.com> FYI, ActiveState and O'Reilly launched the new code.activestate.com with code recipes for dynamic languages such as Python, Perl and Tcl and web development. This site is great recipe sharing site for all Python, Perl and Tcl developers. O'Reilly will be use recipes from the site for its next Python cook book. Bit.ly link to the blog announcement: http://bit.ly/b1Wkdm From cappy2112 at gmail.com Sat Feb 20 06:10:59 2010 From: cappy2112 at gmail.com (Tony Cappellini) Date: Fri, 19 Feb 2010 21:10:59 -0800 Subject: [Baypiggies] Adding gif's to a PDF Message-ID: <8249c4ac1002192110g1385818cnb7488226c71bd69c@mail.gmail.com> I've got several directories of gif images that are scans of an out of print magazines. I'd like to write a program to iterate over the directories & images, and put the gifs as pages into a PDF file. The gifs are full page scans and I want them to show up in the PDF as a full page, not a thumbnail or small image. Iterating over the directories & images is cake, but I've never written a PDF pragmatically. I'll be doing this on Windows. Does anyone have an example of how to write images to a PDF file? thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Sat Feb 20 16:20:37 2010 From: aahz at pythoncraft.com (Aahz) Date: Sat, 20 Feb 2010 07:20:37 -0800 Subject: [Baypiggies] Adding gif's to a PDF In-Reply-To: <8249c4ac1002192110g1385818cnb7488226c71bd69c@mail.gmail.com> References: <8249c4ac1002192110g1385818cnb7488226c71bd69c@mail.gmail.com> Message-ID: <20100220152037.GA12834@panix.com> On Fri, Feb 19, 2010, Tony Cappellini wrote: > > I've got several directories of gif images that are scans of an out > of print magazines. I'd like to write a program to iterate over the > directories & images, and put the gifs as pages into a PDF file. The > gifs are full page scans and I want them to show up in the PDF as a > full page, not a thumbnail or small image. > > Does anyone have an example of how to write images to a PDF file? Dunno about GIFs, but the canonical Python library for writing PDF is http://www.reportlab.com/ -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. :-)" From motoom at xs4all.nl Sat Feb 20 17:04:33 2010 From: motoom at xs4all.nl (Michiel Overtoom) Date: Sat, 20 Feb 2010 17:04:33 +0100 Subject: [Baypiggies] Adding gif's to a PDF In-Reply-To: <8249c4ac1002192110g1385818cnb7488226c71bd69c@mail.gmail.com> References: <8249c4ac1002192110g1385818cnb7488226c71bd69c@mail.gmail.com> Message-ID: <4B800811.7000902@xs4all.nl> On 2010-02-20 06:10, Tony Cappellini wrote: > Does anyone have an example of how to write images to a PDF file? from reportlab.pdfgen import canvas import glob c = canvas.Canvas("book.pdf") for fn in glob.glob("*.gif"): dim = c.drawImage(fn,0,0) c.setPageSize(dim) c.showPage() c.save() Greetings, -- "The ability of the OSS process to collect and harness the collective IQ of thousands of individuals across the Internet is simply amazing." - Vinod Valloppillil http://www.catb.org/~esr/halloween/halloween4.html From cappy2112 at gmail.com Sat Feb 20 17:21:36 2010 From: cappy2112 at gmail.com (Tony Cappellini) Date: Sat, 20 Feb 2010 08:21:36 -0800 Subject: [Baypiggies] Adding gif's to a PDF In-Reply-To: <4B800811.7000902@xs4all.nl> References: <8249c4ac1002192110g1385818cnb7488226c71bd69c@mail.gmail.com> <4B800811.7000902@xs4all.nl> Message-ID: <8249c4ac1002200821u2ee2704bgd3a37d7e71560fb3@mail.gmail.com> Thanks Michael. Since I've posted my message, I've found 3 python libraries for writing PDF files, but their documentation doesn't state anything regarding "adding images". I'll try your code. On Sat, Feb 20, 2010 at 8:04 AM, Michiel Overtoom wrote: > On 2010-02-20 06:10, Tony Cappellini wrote: > > Does anyone have an example of how to write images to a PDF file? >> > > from reportlab.pdfgen import canvas > import glob > c = canvas.Canvas("book.pdf") > for fn in glob.glob("*.gif"): > dim = c.drawImage(fn,0,0) > c.setPageSize(dim) > c.showPage() > c.save() > > > > Greetings, > > -- > "The ability of the OSS process to collect and harness > the collective IQ of thousands of individuals across > the Internet is simply amazing." - Vinod Valloppillil > http://www.catb.org/~esr/halloween/halloween4.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cappy2112 at gmail.com Sun Feb 21 02:11:41 2010 From: cappy2112 at gmail.com (Tony Cappellini) Date: Sat, 20 Feb 2010 17:11:41 -0800 Subject: [Baypiggies] Adding gif's to a PDF In-Reply-To: <4B800811.7000902@xs4all.nl> References: <8249c4ac1002192110g1385818cnb7488226c71bd69c@mail.gmail.com> <4B800811.7000902@xs4all.nl> Message-ID: <8249c4ac1002201711w79f27fc6yf14404e5ba7b3101@mail.gmail.com> Michael, It worked perfectly, verbatim. Not knowing anything about the structure of PDFs, I was envisioning I would need to tweak each image, but the PDF came out exactly as I was expecting! Many thanks On Sat, Feb 20, 2010 at 8:04 AM, Michiel Overtoom wrote: > On 2010-02-20 06:10, Tony Cappellini wrote: > > Does anyone have an example of how to write images to a PDF file? >> > > from reportlab.pdfgen import canvas > import glob > c = canvas.Canvas("book.pdf") > for fn in glob.glob("*.gif"): > dim = c.drawImage(fn,0,0) > c.setPageSize(dim) > c.showPage() > c.save() > > > > Greetings, > > -- > "The ability of the OSS process to collect and harness > the collective IQ of thousands of individuals across > the Internet is simply amazing." - Vinod Valloppillil > http://www.catb.org/~esr/halloween/halloween4.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cappy2112 at gmail.com Sun Feb 21 05:42:05 2010 From: cappy2112 at gmail.com (Tony Cappellini) Date: Sat, 20 Feb 2010 20:42:05 -0800 Subject: [Baypiggies] For those who subscribe to Python Magazine Message-ID: <8249c4ac1002202042n764b6bbx3b03ef15a21a67bb@mail.gmail.com> In case you've been wondering why the website has been offline for so long, and there haven't been any issues in several months http://rhodesmill.org/brandon/2010/leaving-python-magazine/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From p at ulmcnett.com Sun Feb 21 06:37:06 2010 From: p at ulmcnett.com (Paul McNett) Date: Sun, 21 Feb 2010 00:37:06 -0500 Subject: [Baypiggies] Salary Ranges In-Reply-To: <78b3a9581002170847i21754847jc4ee7b42700959a2@mail.gmail.com> References: <78b3a9581002170847i21754847jc4ee7b42700959a2@mail.gmail.com> Message-ID: <4B80C682.5000501@ulmcnett.com> On 2/17/10 11:47 AM, wesley chun wrote: > i'd write a script to keep on submitting to the machine until i don't > get rejected in order to see what value is stored in > tiny_amount_we_want_to_pay. :-) What about just entering $1. And if that fails, keep trying (10, 100, 1000, 10000, 20000, 30000...) In other words, find and submit the minimum amount the form will allow. Then once you've wow'd 'em with your abilities and the talk comes to salary, you can say "obviously I must have made a mistake on the screening form: we are going to have to negotiate that point going forward." At least it gets you past being rejected by the bleeping computer. Paul From keith at dartworks.biz Sun Feb 21 06:40:18 2010 From: keith at dartworks.biz (Keith Dart) Date: Sat, 20 Feb 2010 21:40:18 -0800 Subject: [Baypiggies] For those who subscribe to Python Magazine In-Reply-To: <8249c4ac1002202042n764b6bbx3b03ef15a21a67bb@mail.gmail.com> References: <8249c4ac1002202042n764b6bbx3b03ef15a21a67bb@mail.gmail.com> Message-ID: <20100220214018.100756fa@dartworks.biz> === On Sat, 02/20, Tony Cappellini wrote: === > In case you've been wondering why the website has been offline for so > long, and there haven'tbeen any issues in several months === I didn't even know there was a Python magazine. :-o -- Keith Dart -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: 19017044 ===================================================================== From eric at ericwalstad.com Mon Feb 22 02:24:51 2010 From: eric at ericwalstad.com (Eric Walstad) Date: Sun, 21 Feb 2010 17:24:51 -0800 Subject: [Baypiggies] Salary Ranges In-Reply-To: <4B80C682.5000501@ulmcnett.com> References: <78b3a9581002170847i21754847jc4ee7b42700959a2@mail.gmail.com> <4B80C682.5000501@ulmcnett.com> Message-ID: On Sat, Feb 20, 2010 at 9:37 PM, Paul McNett

wrote: > In other words, find and submit the minimum amount the form will allow. Then > once you've wow'd 'em with your abilities and the talk comes to salary, you > can say "obviously I must have made a mistake on the screening form... But it wouldn't have been a mistake. There's something about this approach that just rubs me the wrong way. I'd rather honestly tell them of the approach taken to get past the bot. Like it or not, they'd have the truth and a bit of understanding about me. Eric From rich at noir.com Mon Feb 22 09:08:37 2010 From: rich at noir.com (K. Richard Pixley) Date: Mon, 22 Feb 2010 00:08:37 -0800 Subject: [Baypiggies] newbie question - json as pickler Message-ID: <4B823B85.3060405@noir.com> I was hoping to use json as a direct pickle replacement for some log files, wire transfers, and state dump/restores, at least for debugging, because it's essentially human readable. But I've run into two problems. Problem #1: most json encoders, (including the standard library json module), don't encode/decode arbitrary objects. They seem to be aiming for cross language exchange rather than persistence or serialization of python-to-python transfers. I have found a couple on the cheese shop that do use a standard encoding, so this isn't necessarily a blocker. Problem #2: the object encoding serializers that I've found are strictly string->object and object->string. This would seem to make a number of things awkward. Spitting out a sequence of objects as a sequence of strings is easy. However, it's looking to me as though reading that sequence of strings will require parsing the incoming stream in order to divide them up into chunks suitable for handing off to the deserializer. This seems like it makes the serializers essentially useless. I feel like I must be missing something here. Does anyone else see it? Is there an obvious way to break up the byte stream? Is there an easy way to tease the standard library coder into coding arbitrary objects? --rich From keith at dartworks.biz Mon Feb 22 09:22:03 2010 From: keith at dartworks.biz (Keith Dart) Date: Mon, 22 Feb 2010 00:22:03 -0800 Subject: [Baypiggies] newbie question - json as pickler In-Reply-To: <4B823B85.3060405@noir.com> References: <4B823B85.3060405@noir.com> Message-ID: <20100222002203.4cace29f@dartworks.biz> === On Mon, 02/22, K. Richard Pixley wrote: === > Problem #1: most json encoders, (including the standard library json > module), don't encode/decode arbitrary objects. They seem to be > aiming for cross language exchange rather than persistence or > serialization of python-to-python transfers. I have found a couple > on the cheese shop that do use a standard encoding, so this isn't > necessarily a blocker. Right. If you want Python-to-Python transfers why not use pickle? The default format IS text, and is actually readable once you learn a little of the format. > Problem #2: the object encoding serializers that I've found are > strictly string->object and object->string. This would seem to make > a number of things awkward. Spitting out a sequence of objects as a > sequence of strings is easy. However, it's looking to me as though > reading that sequence of strings will require parsing the incoming > stream in order to divide them up into chunks suitable for handing > off to the deserializer. This seems like it makes the serializers > essentially useless. I don't think you should be trying to slice the serialized byte stream since it is supposed to be opaque data. You can pickle arbitrary objects. The main limitation there is that class instances don't pickle the whole class, but refer to the class by name. Therefore when you unpickle that instance the class object must be available, and on the same "package path". The exact class type is not even checked, so you can unpickle to a different class implementation than you pickled with. I've exploited this myself when transferring objects between Linux and Windows. ;-) However, if all you want to transfer and/or store are basic data types then JSON serialization could work. It is possible to extend it with a custom hack to transfer more complex types, but it won't be portable. -- Keith Dart -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: 19017044 ===================================================================== From fenn at SDF.LONESTAR.ORG Mon Feb 22 10:00:28 2010 From: fenn at SDF.LONESTAR.ORG (ben lipkowitz) Date: Mon, 22 Feb 2010 09:00:28 +0000 (UTC) Subject: [Baypiggies] newbie question - json as pickler In-Reply-To: <4B823B85.3060405@noir.com> References: <4B823B85.3060405@noir.com> Message-ID: On Mon, 22 Feb 2010, K. Richard Pixley wrote: > I was hoping to use json as a direct pickle replacement for some log files, > wire transfers, and state dump/restores, at least for debugging, because it's > essentially human readable. But I've run into two problems. > > Problem #1: most json encoders, (including the standard library json module), > don't encode/decode arbitrary objects. They seem to be aiming for cross > language exchange rather than persistence or serialization of > python-to-python transfers. I have found a couple on the cheese shop that do > use a standard encoding, so this isn't necessarily a blocker. you might want to look into YAML: http://pyyaml.org/wiki/PyYAMLDocumentation it automatically serializes any pure python objects (no SWIG wrappers etc) also, JSON is valid YAML -fenn From aahz at pythoncraft.com Mon Feb 22 15:50:54 2010 From: aahz at pythoncraft.com (Aahz) Date: Mon, 22 Feb 2010 06:50:54 -0800 Subject: [Baypiggies] newbie question - json as pickler In-Reply-To: <4B823B85.3060405@noir.com> References: <4B823B85.3060405@noir.com> Message-ID: <20100222145054.GA6732@panix.com> On Mon, Feb 22, 2010, K. Richard Pixley wrote: > > Problem #2: the object encoding serializers that I've found are strictly > string->object and object->string. This would seem to make a number of > things awkward. Spitting out a sequence of objects as a sequence of > strings is easy. However, it's looking to me as though reading that > sequence of strings will require parsing the incoming stream in order to > divide them up into chunks suitable for handing off to the deserializer. > This seems like it makes the serializers essentially useless. Just serialize a list of objects. > I feel like I must be missing something here. Does anyone else see it? > Is there an obvious way to break up the byte stream? Is there an easy > way to tease the standard library coder into coding arbitrary objects? You do what pickle essentially does: serialize o.__dict__ -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. :-)" From rich at noir.com Mon Feb 22 16:35:11 2010 From: rich at noir.com (K. Richard Pixley) Date: Mon, 22 Feb 2010 07:35:11 -0800 Subject: [Baypiggies] newbie question - json as pickler In-Reply-To: <20100222002203.4cace29f@dartworks.biz> References: <4B823B85.3060405@noir.com> <20100222002203.4cace29f@dartworks.biz> Message-ID: <4B82A42F.9020502@noir.com> Keith Dart wrote: > Right. If you want Python-to-Python transfers why not use pickle? > The default format IS text, and is actually readable once you learn a > little of the format. > Interesting. I was using pickle.HIGHEST_PROTOCOL which isn't text. Thanks. > I don't think you should be trying to slice the serialized byte stream > since it is supposed to be opaque data. > Exactly. > You can pickle arbitrary objects. The main limitation there is that > class instances don't pickle the whole class, but refer to the class by > name. Understood. > However, if all you want to transfer and/or store are basic data types > then JSON serialization could work. It is possible to extend it with a > custom hack to transfer more complex types, but it won't be portable. Understood. And in my case, that's fine. I'll be the only producer and the only consumer. --rich From rich at noir.com Mon Feb 22 16:41:52 2010 From: rich at noir.com (K. Richard Pixley) Date: Mon, 22 Feb 2010 07:41:52 -0800 Subject: [Baypiggies] newbie question - json as pickler In-Reply-To: <20100222145054.GA6732@panix.com> References: <4B823B85.3060405@noir.com> <20100222145054.GA6732@panix.com> Message-ID: <4B82A5C0.1060304@noir.com> Aahz wrote: > Just serialize a list of objects. > My list isn't finite. I'm trying to serialize a log of incoming messages. It won't be finite until it's done and that may take days or weeks and may not all fit in memory. >> I feel like I must be missing something here. Does anyone else see it? >> Is there an obvious way to break up the byte stream? Is there an easy >> way to tease the standard library coder into coding arbitrary objects? >> > You do what pickle essentially does: serialize o.__dict__ > Thank you. That's about what I'd come to on my own. That or a suitable subset thereof. --rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich at noir.com Mon Feb 22 16:49:08 2010 From: rich at noir.com (K. Richard Pixley) Date: Mon, 22 Feb 2010 07:49:08 -0800 Subject: [Baypiggies] newbie question - json as pickler In-Reply-To: References: <4B823B85.3060405@noir.com> Message-ID: <4B82A774.60401@noir.com> ben lipkowitz wrote: > you might want to look into YAML: > http://pyyaml.org/wiki/PyYAMLDocumentation > > it automatically serializes any pure python objects (no SWIG wrappers > etc) > > also, JSON is valid YAML Thank you! That may be exactly what I need. I'm not sure yaml is "human readable". (J/k). But at least it's well documented. :). --rich From daryl.spitzer at gmail.com Mon Feb 22 16:52:50 2010 From: daryl.spitzer at gmail.com (Daryl Spitzer) Date: Mon, 22 Feb 2010 07:52:50 -0800 Subject: [Baypiggies] newbie question - json as pickler In-Reply-To: <4B82A774.60401@noir.com> References: <4B823B85.3060405@noir.com> <4B82A774.60401@noir.com> Message-ID: IMO the YAML Wikipedia page (http://en.wikipedia.org/wiki/Yaml) provides better documentation. -- Daryl On Mon, Feb 22, 2010 at 7:49 AM, K. Richard Pixley wrote: > ben lipkowitz wrote: >> >> you might want to look into YAML: >> http://pyyaml.org/wiki/PyYAMLDocumentation >> >> it automatically serializes any pure python objects (no SWIG wrappers etc) >> >> also, JSON is valid YAML > > Thank you! ?That may be exactly what I need. > > I'm not sure yaml is "human readable". ?(J/k). ?But at least it's well > documented. ?:). > > --rich > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From roderick at sanfransystems.com Mon Feb 22 19:42:00 2010 From: roderick at sanfransystems.com (Roderick Llewellyn) Date: Mon, 22 Feb 2010 10:42:00 -0800 Subject: [Baypiggies] Salary Ranges References: Message-ID: The ideas about getting past bots by doing scripts to repost, etc, are really clever, but they might not work. The bot might well say "already submitted under this name" or the like. I suppose you could make up names for each trial til you found the magic number that works, then submit under your real name. I think though, you might get past the bot this way, but the first human that actually looked at these multiple submissions would kick you out for trying to game their system. They're trying to game you -- they don't want you to game them. As one of the writers pointed out, companies like Facebook would much rather have false negatives (bouncing qualified people) than false positives (hiring incompetents). That will change when the economy moves around and the pool of unemployed qualified candidates dries up. You'll stop seeing the job postings with 'tude and the endless robotic hurdles to jump over. Oh for the Clinton days again!!! :) From aahz at pythoncraft.com Mon Feb 22 20:11:40 2010 From: aahz at pythoncraft.com (Aahz) Date: Mon, 22 Feb 2010 11:11:40 -0800 Subject: [Baypiggies] Ban recruiters again? In-Reply-To: References: Message-ID: <20100222191140.GA13560@panix.com> On Mon, Feb 22, 2010, Roderick Llewellyn wrote: > > As one of the writers pointed out, companies like Facebook would much > rather have false negatives (bouncing qualified people) than false > positives (hiring incompetents). That will change when the economy moves > around and the pool of unemployed qualified candidates dries up. You'll > stop seeing the job postings with 'tude and the endless robotic hurdles > to jump over. Oh for the Clinton days again!!! :) Actually, my impression is that the pool of unemployed QUALIFIED candidates has already dried up around here. I was thinking this weekend that it's time to re-enable the ban on recruiters. Any opinions? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer From rich at noir.com Mon Feb 22 20:22:54 2010 From: rich at noir.com (K. Richard Pixley) Date: Mon, 22 Feb 2010 11:22:54 -0800 Subject: [Baypiggies] Ban recruiters again? In-Reply-To: <20100222191140.GA13560@panix.com> References: <20100222191140.GA13560@panix.com> Message-ID: <4B82D98E.4050705@noir.com> Aahz wrote: > Any opinions? My opinion is that there hasn't been enough traffic to warrant discussion. Besides, look at the entertainment value we got from the last one! :). --rich From keith at dartworks.biz Mon Feb 22 20:27:41 2010 From: keith at dartworks.biz (Keith Dart) Date: Mon, 22 Feb 2010 11:27:41 -0800 Subject: [Baypiggies] Ban recruiters again? In-Reply-To: <20100222191140.GA13560@panix.com> References: <20100222191140.GA13560@panix.com> Message-ID: <20100222112741.35a0e378@dartworks.biz> === On Mon, 02/22, Aahz wrote: === > Actually, my impression is that the pool of unemployed QUALIFIED > candidates has already dried up around here. I was thinking this > weekend that it's time to re-enable the ban on recruiters. Any > opinions? === I don't think we can really know that, now or in the future. We also don't know if there are any currently employed that might also be looking. So I vote for keeping Python related job postings. -- Keith Dart -- -- -------------------- Keith Dart ======================= From roderick at sanfransystems.com Mon Feb 22 20:54:26 2010 From: roderick at sanfransystems.com (Roderick Llewellyn) Date: Mon, 22 Feb 2010 11:54:26 -0800 Subject: [Baypiggies] Salary etc. References: Message-ID: Aahz says: "Actually, my impression is that the pool of unemployed QUALIFIED candidates has already dried up around here. I was thinking this weekend that it's time to re-enable the ban on recruiters. Any opinions?" Yes I think you're right. I'm getting actively courted by many internal and external recruiters now and likely will have > 1 offer very soon! I'm not getting the sneering kind of responses I was getting last year especially once they knew I am not GenX. I no longer apply to generic Web/LAMP "download much... develop little" type of jobs however, I realized they were lost causes, as those places seem religious in tone and don't want you to have worked on anything BUT web technologies, want you to be an open source contributor (i.e., be part of the choir)... and they DEFINITELY don't want to hire Baby Boomers. As I've already said before. The trick was to find more specialized companies that need more than just generic web programmers. Also I think the economy is heating up again tho more slowly than one might like. About banning recruiters from this list: If they are describing a specific job that seems real (not just resume-fishing with phony jobs as many do) and Python-related, it seems like if it leads to one person getting a job and feeding his/her family that's a good thing. If they're just spamming "check out my jobs and... of course... send those resumes in" I'd cut them. Roderick Llewellyn From spmcinerney at hotmail.com Mon Feb 22 21:01:50 2010 From: spmcinerney at hotmail.com (Stephen McInerney) Date: Mon, 22 Feb 2010 12:01:50 -0800 Subject: [Baypiggies] Ban recruiters again? In-Reply-To: <20100222191140.GA13560@panix.com> References: , , <20100222191140.GA13560@panix.com> Message-ID: No, don't ban recruiters. Just remove them individually if they repeat violations (non-Pythonic/ non-local). Stephen _________________________________________________________________ Hotmail: Trusted email with powerful SPAM protection. http://clk.atdmt.com/GBL/go/201469227/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From simeonf at gmail.com Mon Feb 22 21:06:44 2010 From: simeonf at gmail.com (Simeon Franklin) Date: Mon, 22 Feb 2010 12:06:44 -0800 Subject: [Baypiggies] Ban recruiters again? In-Reply-To: <20100222191140.GA13560@panix.com> References: <20100222191140.GA13560@panix.com> Message-ID: I was under the impression we were still banning recruiters - I've pointed a couple of recent posters to our Jobs Policy which still says: Principals only, no recruiters... http://www.baypiggies.net/index_html/job-listings -regards Simeon Franklin On Mon, Feb 22, 2010 at 11:11 AM, Aahz wrote: > On Mon, Feb 22, 2010, Roderick Llewellyn wrote: >> >> As one of the writers pointed out, companies like Facebook would much >> rather have false negatives (bouncing qualified people) than false >> positives (hiring incompetents). That will change when the economy moves >> around and the pool of unemployed qualified candidates dries up. You'll >> stop seeing the job postings with 'tude and the endless robotic hurdles >> to jump over. Oh for the Clinton days again!!! :) > > Actually, my impression is that the pool of unemployed QUALIFIED > candidates has already dried up around here. ?I was thinking this weekend > that it's time to re-enable the ban on recruiters. ?Any opinions? > -- > Aahz (aahz at pythoncraft.com) ? ? ? ? ? <*> ? ? ? ? http://www.pythoncraft.com/ > > "Many customs in this life persist because they ease friction and promote > productivity as a result of universal agreement, and whether they are > precisely the optimal choices is much less important." --Henry Spencer > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From luca.pellicoro at gmail.com Mon Feb 22 21:08:24 2010 From: luca.pellicoro at gmail.com (Luca Pellicoro) Date: Mon, 22 Feb 2010 12:08:24 -0800 Subject: [Baypiggies] Ban recruiters again? In-Reply-To: <20100222112741.35a0e378@dartworks.biz> References: <20100222191140.GA13560@panix.com> <20100222112741.35a0e378@dartworks.biz> Message-ID: On Mon, Feb 22, 2010 at 11:27 AM, Keith Dart wrote: > === On Mon, 02/22, Aahz wrote: === >> Actually, my impression is that the pool of unemployed QUALIFIED >> candidates has already dried up around here. ?I was thinking this >> weekend that it's time to re-enable the ban on recruiters. ?Any >> opinions? > === > > I don't think we can really know that, now or in the future. We also > don't know if there are any currently employed that might also be > looking. So I vote for keeping Python related job postings. > I second that. About half-an-hour after I posted the last job on this list looking for a python loving co-worker, our HR team received a surge of 30 resumes, but no on-or-off list comments. Whether we received qualified applicants or not still needs to be determined. From aahz at pythoncraft.com Mon Feb 22 21:29:55 2010 From: aahz at pythoncraft.com (Aahz) Date: Mon, 22 Feb 2010 12:29:55 -0800 Subject: [Baypiggies] Ban recruiters again? In-Reply-To: <20100222112741.35a0e378@dartworks.biz> References: <20100222191140.GA13560@panix.com> <20100222112741.35a0e378@dartworks.biz> Message-ID: <20100222202955.GA5473@panix.com> On Mon, Feb 22, 2010, Keith Dart wrote: > === On Mon, 02/22, Aahz wrote: === >> >> Actually, my impression is that the pool of unemployed QUALIFIED >> candidates has already dried up around here. I was thinking this >> weekend that it's time to re-enable the ban on recruiters. Any >> opinions? > > I don't think we can really know that, now or in the future. We also > don't know if there are any currently employed that might also be > looking. So I vote for keeping Python related job postings. Job postings from companies have always been welcome (although a vocal minority occasionally agitates to push them to a separate list). The only question is whether we return to banning recruiters not associated with a specific company. (We voted to temporarily allow recruiters due to the recession.) There are three reasons that I know of for banning recruiters: * They are more likely to spam * They are less likely to participate in discussion * Duplicate ads Recruiters often try to argue about whether they're breaking the first two points when you ban them for behavior, so it's usually easier to just ban recruiters, because even the good ones run up against the last point. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer From laban.patrick at gmail.com Mon Feb 22 21:42:09 2010 From: laban.patrick at gmail.com (Patrick Laban) Date: Mon, 22 Feb 2010 12:42:09 -0800 Subject: [Baypiggies] Fwd: Salary etc. In-Reply-To: <5ee958ff1002221241x1dddaa19o9c5d9e825ca57141@mail.gmail.com> References: <5ee958ff1002221241x1dddaa19o9c5d9e825ca57141@mail.gmail.com> Message-ID: <5ee958ff1002221242h57aaeb0dtb6c9d7eaf372c133@mail.gmail.com> ---------- Forwarded message ---------- From: Patrick Laban Date: Mon, Feb 22, 2010 at 12:41 PM Subject: Re: [Baypiggies] Salary etc. To: Roderick Llewellyn I would like to think I'm a unemployed qualified candidate, at least at the entry level. My experience so far with recruiters is that almost all of them won't put any serious effort into working with an entry level candidate, even if the position is hiring at several experience levels. I've also had a recruiter quit while she was presenting me to another company. On Mon, Feb 22, 2010 at 11:54 AM, Roderick Llewellyn < roderick at sanfransystems.com> wrote: > Aahz says: "Actually, my impression is that the pool of unemployed > QUALIFIED > candidates has already dried up around here. I was thinking this weekend > that it's time to re-enable the ban on recruiters. Any opinions?" > > Yes I think you're right. I'm getting actively courted by many internal and > external recruiters now and likely will have > 1 offer very soon! I'm not > getting the sneering kind of responses I was getting last year especially > once they knew I am not GenX. I no longer apply to generic Web/LAMP > "download much... develop little" type of jobs however, I realized they were > lost causes, as those places seem religious in tone and don't want you to > have worked on anything BUT web technologies, want you to be an open source > contributor (i.e., be part of the choir)... and they DEFINITELY don't want > to hire Baby Boomers. As I've already said before. The trick was to find > more specialized companies that need more than just generic web programmers. > Also I think the economy is heating up again tho more slowly than one might > like. > > About banning recruiters from this list: If they are describing a specific > job that seems real (not just resume-fishing with phony jobs as many do) and > Python-related, it seems like if it leads to one person getting a job and > feeding his/her family that's a good thing. If they're just spamming "check > out my jobs and... of course... send those resumes in" I'd cut them. > > Roderick Llewellyn > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob at redivi.com Tue Feb 23 00:13:25 2010 From: bob at redivi.com (Bob Ippolito) Date: Mon, 22 Feb 2010 15:13:25 -0800 Subject: [Baypiggies] [JOB] Mochi Media, Inc. hiring in San Francisco, CA Message-ID: <6a36e7291002221513q407936bds4a3cc4dc6b7a0d2e@mail.gmail.com> Mochi Media, Inc. is hiring immediately for several positions in San Francisco, CA to help grow our game platform empire. The UI Engineer position is primarily Pylons/Django and the Platform Engineer position is Python and Erlang. While the "requirements" may not fit your resume exactly we're really just interested in growing our team with more great people, so if you're interested you should definitely drop us a line here: http://www.mochimedia.com/jobs.html If you're currently at the PyCon 2010 sprints in Atlanta and would like to speak with a Live Mochi you can get in touch with David Reid (@dreid) or Eric Florenzano (@ericflo). More general information about Mochi Media is at http://www.mochimedia.com/ and if you'd like to speak to a human immediately you can email me at bob at mochimedia.com -- I'm CTO and cofounder so I'd be more than happy to answer any questions that you may have :) -bob From keith at dartworks.biz Tue Feb 23 00:41:04 2010 From: keith at dartworks.biz (Keith Dart) Date: Mon, 22 Feb 2010 15:41:04 -0800 Subject: [Baypiggies] [JOB] Mochi Media, Inc. hiring in San Francisco, CA In-Reply-To: <6a36e7291002221513q407936bds4a3cc4dc6b7a0d2e@mail.gmail.com> References: <6a36e7291002221513q407936bds4a3cc4dc6b7a0d2e@mail.gmail.com> Message-ID: <20100222154104.49db1eff@dartworks.biz> === On Mon, 02/22, Bob Ippolito wrote: === > More general information about Mochi Media is at > http://www.mochimedia.com/ and if you'd like to speak to a human > immediately you can email me at bob at mochimedia.com -- I'm CTO and > cofounder so I'd be more than happy to answer any questions that you > may have :) === Bob is also the MochiKit God. Thanks Bob! -- Keith Dart -- -- -------------------- Keith Dart ======================= From craigfrazersmith at gmail.com Tue Feb 23 01:36:38 2010 From: craigfrazersmith at gmail.com (Craig Smith) Date: Mon, 22 Feb 2010 16:36:38 -0800 Subject: [Baypiggies] If a picture is worth a thousand words #pycon is worth 6918 tweets. Message-ID: <4B832316.4070903@gmail.com> Hi: I was following pycon on twitter this weekend and realized there was no way I could keep up with all the tweets going by. Also a twitter robo-troll (welcome to the future) was irritating me more than it should have. So yesterday I downloaded all the #pycon tagged tweets and pulled out just the ones that had been retweeted by other people figuring those were the most interesting. My theory is that the more something is retweeted the more interesting it should be. Without further ado the top 10 tweets of #pycon 2010 were: 10 - RT @gvanrossum: tinyurl.com/pypyslides #pycon #pypy 10 - RT @VanL: The first video from #pycon 2010 just hit http://pycon.blip.tv! 11 - RT @dabeaz: Understanding the Python GIL Slides and demo are up. http://www.dabeaz.com/GIL #pycon 12 - RT @mpirnat: "Postgres is available if you need a database; MySQL is available if you don't." -- Brandon Rhodes at #pycon talk 13 - RT @gvanrossum: BDFL etiquette: Saying hi shaking hands: yes. Photo-op: no. Friendly chat: yes. Idea pitch > 15 seconds: expect me to get rude. #pycon 16 - RT @spyced: "What every developer should know about database scalability" #pycon slides aka the case for #nosql: http://bit.ly/96fFZ0 16 - RT @gvanrossum: Test your web apps with Ping's scrape.py: http://bit.ly/dhQdx6 #pycon // looking freaking great 23 - RT @voidspace: YouTube is the second largest search site on the internet and is pure Python. #pycon 47 - RT @gvanrossum: The Hitchhiker?s Guide to Packaging - http://guide.python-distribute.org/ #python #pycon 63 - RT @gvanrossum: Unladen Swallow is a go! #pycon #python -Craig Smith p.s. These numbers are fairly accurate, but I'm not going to guarantee that my regex caught all the retweets perfectly and I didn't handle retweets of retweets well. This was afternoon goofing off with Python, not production code. Caveat emptor. p.p.s More fun with numbers: Total tweets tagged #pycon from Wednesday morning-ish to Sunday afternoon-ish: 6918 Number of unique usernames making those tweets: 1268. Number of retweets (a subset of the total number) 1304. Most retweeted people with counts: (u'DanielMcGraw', 9) (u'annaraven', 9) (u'bradallen137', 9) (u'dhdspeech', 9) (u'ieure', 9) (u'tsaylor', 9) (u'Matt_Wilbanks', 10) (u'jeremyhylton', 10) (u'tarek_ziade', 10) (u'joestump', 11) (u'djangopony', 12) (u'holdenweb', 12) (u'pydanny', 12) (u'activestate', 13) (u'natea', 13) (u'dabeaz', 14) (u'ericholscher', 15) (u'raymondh', 15) (u'jessenoller', 17) (u'justinlilly', 17) (u'VanL', 20) (u'garnaat', 20) (u'johnderosa', 20) (u'spyced', 22) (u'mpirnat', 26) (u'jacobian', 31) (u'voidspace', 65) (u'gvanrossum', 191) People who made the most tweets tagged #pycon: (u'gsiegman', 41) (u'kevbo', 41) (u'lukesneeringer', 41) (u'b3ng0', 42) (u'griggheo', 42) (u'mpirnat', 43) (u'wescpy', 43) (u'bitprophet', 46) (u'swheatley', 46) (u'alfredodeza', 47) (u'jessenoller', 52) (u'voidspace', 52) (u'petrillic', 54) (u'alex_gaynor', 55) (u'ryazwinski', 55) (u'edleafe', 57) (u'kcunning', 57) (u'roachmj', 57) (u'joshourisman', 58) (u'offby1', 58) (u'elpargo', 68) (u'benjaminws', 70) (u'pydanny', 71) (u'mikemaccana', 81) (u'dhdspeech', 83) (u'annaraven', 86) (u'PyconPads', 88) (u'johnderosa', 97) (u'twatotron', 185) <--- The irritating Bot. From rob at kalistra.com Tue Feb 23 01:51:29 2010 From: rob at kalistra.com (Rob Miller) Date: Mon, 22 Feb 2010 16:51:29 -0800 Subject: [Baypiggies] Ban recruiters again? In-Reply-To: <20100222191140.GA13560@panix.com> References: <20100222191140.GA13560@panix.com> Message-ID: Aahz wrote: > On Mon, Feb 22, 2010, Roderick Llewellyn wrote: >> As one of the writers pointed out, companies like Facebook would much >> rather have false negatives (bouncing qualified people) than false >> positives (hiring incompetents). That will change when the economy moves >> around and the pool of unemployed qualified candidates dries up. You'll >> stop seeing the job postings with 'tude and the endless robotic hurdles >> to jump over. Oh for the Clinton days again!!! :) > > Actually, my impression is that the pool of unemployed QUALIFIED > candidates has already dried up around here. As a long-time Python hacker who was laid off a couple of weeks ago and is currently looking for my next gig, I'd be inclined to disagree... ;) That said, I don't feel strongly either way about reinstating the recruiter ban right now. I think the ban is a good idea in general, and if recruiter posts were becoming common I'd definitely want to reinstate it, but for now it doesn't seem like too much of a problem. -r From dalke at dalkescientific.com Tue Feb 23 02:56:56 2010 From: dalke at dalkescientific.com (Andrew Dalke) Date: Mon, 22 Feb 2010 20:56:56 -0500 Subject: [Baypiggies] If a picture is worth a thousand words #pycon is worth 6918 tweets. In-Reply-To: <4B832316.4070903@gmail.com> References: <4B832316.4070903@gmail.com> Message-ID: <84279C2F-1439-4DF5-85AF-97BF5563B325@dalkescientific.com> You were posted. On Feb 22, 2010, at 7:36 PM, Craig Smith wrote: > Hi: > > I was following pycon on twitter this weekend and realized there was no way I could keep up with all the tweets going by. Also a twitter robo-troll (welcome to the future) was irritating me more than it should have. Andrew dalke at dalkescientific.com From dalke at dalkescientific.com Tue Feb 23 02:58:04 2010 From: dalke at dalkescientific.com (Andrew Dalke) Date: Mon, 22 Feb 2010 20:58:04 -0500 Subject: [Baypiggies] If a picture is worth a thousand words #pycon is worth 6918 tweets. In-Reply-To: <84279C2F-1439-4DF5-85AF-97BF5563B325@dalkescientific.com> References: <4B832316.4070903@gmail.com> <84279C2F-1439-4DF5-85AF-97BF5563B325@dalkescientific.com> Message-ID: On Feb 22, 2010, at 8:56 PM, Andrew Dalke wrote: > You were posted. Err, umm, oops! Andrew dalke at dalkescientific.com From keith at dartworks.biz Tue Feb 23 03:39:39 2010 From: keith at dartworks.biz (Keith Dart) Date: Mon, 22 Feb 2010 18:39:39 -0800 Subject: [Baypiggies] If a picture is worth a thousand words #pycon is worth 6918 tweets. In-Reply-To: <4B832316.4070903@gmail.com> References: <4B832316.4070903@gmail.com> Message-ID: <20100222183939.4195a99c@dartworks.biz> === On Mon, 02/22, Craig Smith wrote: === > I was following pycon on twitter this weekend and realized there was > no way I could keep up with all the tweets going by. Also a twitter > robo-troll (welcome to the future) was irritating me more than it > should have. So yesterday I downloaded all the #pycon tagged tweets > and pulled out just the ones that had been retweeted by other people > figuring those were the most interesting. My theory is that the more > something is retweeted the more interesting it should be. === Hey, that sounds like tweet-rank! You should start a company... -- Keith Dart -- -- -------------------- Keith Dart ======================= From aahz at pythoncraft.com Tue Feb 23 05:01:18 2010 From: aahz at pythoncraft.com (Aahz) Date: Mon, 22 Feb 2010 20:01:18 -0800 Subject: [Baypiggies] [ADMIN] Sending to BayPIGgies Message-ID: <20100223040118.GB7797@panix.com> This is just a friendly reminder that ALL e-mail addresses you use to send to BayPIGgies MUST be subscribed. You can set all but one address no-mail to avoid duplicates. We've received a substantial increase recently in posts from non-subscribed addresses, not sure why. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer From tony.loeser at gmail.com Tue Feb 23 09:34:10 2010 From: tony.loeser at gmail.com (Tony Loeser) Date: Tue, 23 Feb 2010 00:34:10 -0800 Subject: [Baypiggies] [Job] Syapse Inc. hiring Python / Django expert Message-ID: <7A57BC8A-F8FE-40B5-A93B-683BD23A6CC0@gmail.com> Syapse is a biomedical informatics startup formed at Stanford and based in Palo Alto, building applications that organize the data in modern biopharma companies. We leverage semantic technologies and biomedical ontologies to deliver scientifically intelligent search, collaboration, and management applications. We are growing, and need to add some key players to the team. In particular, we are looking for a Django expert (or expert in the making) to take over the application delivery platform. This is a ground-level startup opportunity with plenty of responsibility, joining an enthusiastic team with successful entrepreneurs. Key Responsibilities: - Lead the web application development for Syapse's main products and internal tools. - Organize a small team of engineers and designers contributing to the application. - Plan and implement new application features, as well as tools to enable internal operations. - Assure scalability including database and physical web server configuration. - Manage the virtual data center (in the Amazon EC2 cloud). - Optimize and tune web application performance. - Optimize and manage the MySQL database. Requirements: - Minimum of five years experience developing web applications, with experience across the full stack, browser to database. - Fluent in Python, SQL, HTML, Javascript. - Knowledge of standard web application issues including security, performance, caching, HTTP protocol, Apache configuration. - Experience with Django or similar web application framework. - Solid MySQL experience. - Solid documentation, testing, and process fundamentals. - Ability to work independently, collaborate remotely, and bring other engineers up to speed on your work rapidly and thoroughly. - AJAX abilities are a plus. This and other jobs are listed online: http://www.syapse.com/jobs.php#j1 . You can submit a resume through the website, or else contact me directly for a technical discussion at tony-at-syapse.com . -------------- next part -------------- An HTML attachment was scrubbed... URL: From glen at glenjarvis.com Wed Feb 24 03:23:30 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Tue, 23 Feb 2010 18:23:30 -0800 Subject: [Baypiggies] More dramatic material? Message-ID: I've been wanting to prepare materials for regular expressions for a very long time. I'd habitually done what I needed without them. And, because I could get by, it was a crutch. A Linux/Unix certification program at UC Berkeley has pushed me along this way too (using ed, sed, awk, etc.) Regardless, I'm trying to come up with a dramatic example of why to use regular expressions. I imagined to see a hundred-fold increase in the regular expression numbers. However, I've gotten only about twice the efficiency thus far. I started with the Guttenberg Project download of War and Peace. I thought, that's gotta be a large amount of text. Here I use a pythonic, easy to read, simple example to count. We assume that there is no word 'the' spanning across the end-of-one line and started on a new line. Also, this is slightly unfair in that I'm reading one line at a time instead of the whole buffer together (more on this later): f = open("./war_and_peace.txt", 'r') total = 0 for line in f.readlines(): total = total + line.upper().count("THE") print "Total:", total f.close() Several runs on my laptop show this typical type of response: real 0m0.203s user 0m0.176s sys 0m0.023s I was disappointed it was so fast. Where was the drama in that :) Regardless, i did the same run with the following regular expressions. Now, I didn't see that split let me do a good case insensitive search (see comment in code below), so it could be this could be dramatically sped up by using this approach: import re f = open("./war_and_peace.txt", 'r') contents = f.read() # I didn't see I could do case insensitive searching with this flag #m = re.split(r'the', contents, re.I) m = re.split(r'[Tt][Hh][Ee]', contents) print len(m)-1 I get the same actual results, but in the following time (and see this on average). real 0m0.154s user 0m0.124s sys 0m0.026s Well, that's not *that* much of an increase.. Good old python - already pretty fast (and in my mind *much* easier to read than regular expressions). I tried to do something more dramatic -- and avoid the upper case issue to keep the example more fair. I downloaded the FASTA Format of the Human Genome (Chromosome 1) from this address: ftp://ftp.ncbi.nih.gov/genomes/H_sapiens/CHR_01/ Although it's not really a sensical Biological search (mutations, inserts, deletions, etc. keep this from being as 'clean' as regular computer science), I wanted to just be dramatic. So, I searched for 'TGGCCC' in both files (again, we'll ignore any end of line boundaries -- just trying to show the speed performance). Again, repetition gives very similar numbers. Using good old pure python (removing the 'upper()': real 0m5.268s user 0m4.474s sys 0m0.715s Using the equivalent of a fast regular expression (no special matching needed): m = re.split(r'TGGCCC', contents) we get the same results and in time: real 0m5.118s user 0m2.702s sys 0m1.214s Now, we're looking at a large increase. But, again, a factor of about two or less? Can you think of a better example than this? Something more 'wow'.. of course we can multiply the numbers by putting in a huge for loop? but, I was hoping for something more straight forward and obvious why regular expressions are so fast? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich at noir.com Wed Feb 24 04:13:24 2010 From: rich at noir.com (K. Richard Pixley) Date: Tue, 23 Feb 2010 19:13:24 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: References: Message-ID: <4B849954.8050602@noir.com> Regular expressions aren't particularly fast. That's not the reason for using them. In fact, there was a sort of competition going back in the 80's with various types of grep's. Look up boyer-moore, (bgrep), and the difference between egrep and fgrep. The bottom line is that for certain kinds of expressions, different algorithms are more efficient. But the more efficient algorithms trade robustness and problem domain for speed. That is, they work faster, but only on a subset of cases. On the others, they either don't work at all or silently produce incorrect results. Similarly, LALR(0) languages are easier and faster to parse than LALR(1). You don't really need a parser generator like yacc, bison, or byacc if your language is LALR(0). There's a related, but somewhat more subdued competition going on right now about binary diff programs, which have many related properties. So do compression programs and I think everyone's probably aware of that long standing competition. And I'll only mention message digesting, (aka, encryption), which also has similarities. Anything done in C is faster than anything done in python, (assuming competent implementations). The exceptions being things where "pure" python is implemented in C anyway. The win to regular expressions is not speed, but rather that they can search for and find things that other types of searching can't, or can't do easily. For "dramatic" examples, try searching for palindromes, or repeated phrases like 'moinmoin' or 'wikiwiki', or sequences of double repeating characters like vaccuum. --rich From grayarea at reddagger.org Wed Feb 24 04:16:50 2010 From: grayarea at reddagger.org (John Withers) Date: Tue, 23 Feb 2010 19:16:50 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: References: Message-ID: <1266981410.13081.168.camel@Frank-Brain> On Tue, 2010-02-23 at 18:23 -0800, Glen Jarvis wrote: > this way too (using ed, sed, awk, etc.) Regardless, I'm trying to > come up with a dramatic example of why to use regular expressions. I > imagined to see a hundred-fold increase in the regular expression > numbers. However, I've gotten only about twice the efficiency thus > far. > Yeah, I am not sure why you expected that. Someone can correct me if I am wrong, but last time I checked, the engine that python is using for regexes isn't very fast. But to be fair, all modern regex engines are fairly slow compared to theoretical limits or even fast implementations of real formal regular expressions. Because the stuff we are using now and calling regular expressions are really regular expressions with extensions. The extensions, in the form of look ahead and look behind assertions, change the algo that has to be used and massively degrades the perfomance. This is why gnu emacs won't add "modern" features to their regex engines and why you have to specifically flag grep to get them, because it switches in the slower stuff. On the other hand, the simple string handling functions are very highly optimized because they know you aren't going to suddenly throw them a back reference. This would be why throughout python documentation and speed up tips and so on there are many admonitions to use the built in string handling capabilities when ever it makes sense because they are very fast and optimized for what they do. You can find many more details of all this in these various articles: http://lambda-the-ultimate.org/node/2064 http://swtch.com/~rsc/regexp/regexp1.html http://regex.info/blog/2006-09-15/247 http://en.wikipedia.org/wiki/Regular_expression And I have all this information handy because I just had my ass handed to me in a discussion about why gnu emacs should add lookahead/behind assertions :) john withers From spmcinerney at hotmail.com Wed Feb 24 09:18:19 2010 From: spmcinerney at hotmail.com (Stephen McInerney) Date: Wed, 24 Feb 2010 00:18:19 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <4B849954.8050602@noir.com> References: , <4B849954.8050602@noir.com> Message-ID: Rich - another enormous advantage of regexes is simply that they keep code very portable between languages and platform - from Gnu make on WinNT to Python on Mac to UNIX shell etc. Maintainability matters. Stephen > The win to regular expressions is not speed, but rather that they can > search for and find things that other types of searching can't, or can't > do easily. _________________________________________________________________ Hotmail: Free, trusted and rich email service. http://clk.atdmt.com/GBL/go/201469228/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From damonmc at gmail.com Wed Feb 24 12:47:32 2010 From: damonmc at gmail.com (Damon McCormick) Date: Wed, 24 Feb 2010 03:47:32 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: References: Message-ID: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> When searching for a simple string literal, there's no reason that a non-regex function like Python's find(), count(), or index() need be slower than a regex function. If it's slower, this just indicates a potential to further optimize the implementation. A _War and Peace_ search where regular expressions would shine more is this one, which finds all words which contain four vowels in a row: text = open("WnP.txt").read() # in production code, we'd close the file! words = re.findall("\w*[aeiou]{4}\w*", text, re.I) unique_words = set(words) Using pure Python, you'd be hard-pressed to find something as concise or fast that does not use regular expressions. If some of the extended regular expression syntax seems too cryptic, you might prefer an alternative style that hierarchically builds up your expression from sub-expressions, which are in turn built from basic regex constructs: optional_letters = "[A-Za-z]*" vowel = "[AEIOUaeiou]" pattern = optional_letters + 4*vowel + optional_letters unique_words = set(re.findall(pattern, text)) In your chromosome 1 example, you should again be able to see regular expressions shine when searching for a more general pattern. For instance, you might want to find long runs of a given base, using a pattern like "GGGGG+". import gzip text = gzip.open("chr1.fa.gz").read() runs = re.findall("GGGGG+", text) longest_run = max(runs, key=len) -Damon On Tue, Feb 23, 2010 at 6:23 PM, Glen Jarvis wrote: > I've been wanting to prepare materials for regular expressions for a very > long time. I'd habitually done what I needed without them. And, because I > could get by, it was a crutch. > > A Linux/Unix certification program at UC Berkeley has pushed me along this > way too (using ed, sed, awk, etc.) Regardless, I'm trying to come up with a > dramatic example of why to use regular expressions. I imagined to see a > hundred-fold increase in the regular expression numbers. However, I've > gotten only about twice the efficiency thus far. > > I started with the Guttenberg Project download of War and Peace. I thought, > that's gotta be a large amount of text. > > Here I use a pythonic, easy to read, simple example to count. We assume > that there is no word 'the' spanning across the end-of-one line and started > on a new line. Also, this is slightly unfair in that I'm reading one line at > a time instead of the whole buffer together (more on this later): > > f = open("./war_and_peace.txt", 'r') > > total = 0 > for line in f.readlines(): > total = total + line.upper().count("THE") > > print "Total:", total > f.close() > > Several runs on my laptop show this typical type of response: > real 0m0.203s > user 0m0.176s > sys 0m0.023s > > I was disappointed it was so fast. Where was the drama in that :) > > Regardless, i did the same run with the following regular expressions. Now, > I didn't see that split let me do a good case insensitive search (see > comment in code below), so it could be this could be dramatically sped up by > using this approach: > > import re > > f = open("./war_and_peace.txt", 'r') > contents = f.read() > > # I didn't see I could do case insensitive searching with this flag > #m = re.split(r'the', contents, re.I) > m = re.split(r'[Tt][Hh][Ee]', contents) > > print len(m)-1 > > I get the same actual results, but in the following time (and see this on > average). > real 0m0.154s > user 0m0.124s > sys 0m0.026s > > Well, that's not *that* much of an increase.. Good old python - already > pretty fast (and in my mind *much* easier to read than regular expressions). > > I tried to do something more dramatic -- and avoid the upper case issue to > keep the example more fair. > > I downloaded the FASTA Format of the Human Genome (Chromosome 1) from this > address: > > ftp://ftp.ncbi.nih.gov/genomes/H_sapiens/CHR_01/ > > > Although it's not really a sensical Biological search (mutations, inserts, > deletions, etc. keep this from being as 'clean' as regular computer > science), I wanted to just be dramatic. So, I searched for 'TGGCCC' in both > files (again, we'll ignore any end of line boundaries -- just trying to show > the speed performance). > > Again, repetition gives very similar numbers. Using good old pure python > (removing the 'upper()': > real 0m5.268s > user 0m4.474s > sys 0m0.715s > > Using the equivalent of a fast regular expression (no special matching > needed): > > m = re.split(r'TGGCCC', contents) > > we get the same results and in time: > > real 0m5.118s > user 0m2.702s > sys 0m1.214s > > Now, we're looking at a large increase. But, again, a factor of about two > or less? Can you think of a better example than this? Something more 'wow'.. > of course we can multiply the numbers by putting in a huge for loop? but, I > was hoping for something more straight forward and obvious why regular > expressions are so fast? > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Wed Feb 24 15:38:19 2010 From: aahz at pythoncraft.com (Aahz) Date: Wed, 24 Feb 2010 06:38:19 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> Message-ID: <20100224143818.GA23116@panix.com> The other important thing to remember about regular expressions is that even if you use verbose regexes split across multiple lines, regexes are functionally write-only for even the vast majority of experienced programmers. (I.e. almost nobody can look at a complicated regex and understand what it does.) Remember: 'Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.' --Jamie Zawinski Sometimes regexes are the best way to solve a problem, but you really are best served by trying other options first. In many cases, people want to use regexes when they should be using a full-blown parser because regexes require less up-front investment. Just remember that you'll get into technical debt and pay more later. http://en.wikipedia.org/wiki/Technical_debt -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer From glen at glenjarvis.com Wed Feb 24 16:04:06 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Wed, 24 Feb 2010 07:04:06 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <20100224143818.GA23116@panix.com> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> Message-ID: <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> > The other important thing to remember about regular expressions is > that > even if you use verbose regexes split across multiple lines, regexes > are > functionally write-only for even the vast majority of experienced > programmers. (I.e. almost nobody can look at a complicated regex and > understand what it does.) I agree. Regex's have been my weakness forever because it just isn't readable like python is readable. They just feel so unpythonic. I'm always trying to fill in the fundamentals and I've been learning to do fairly sophisticated regex's the past six months... However, my fundamental question of 'why use it on Python' has been echoing in my mind more and more with this conversation. Richard Pixley gave a few reasons other than performance. I'm hoping there will be die hard fans somewhere here to give compelling reasons to learn it. I know we can find them in the Perl camps -- that's logical considering that Perl grew out of a desire to supplement awk. But, it's also a Python module.... So someone must be a die-hard and can defend 'why do it in Python' and give examples. Cheers, Glen > > Remember: > > 'Some people, when confronted with a problem, think "I know, I'll use > regular expressions." Now they have two problems.' --Jamie Zawinski > > Sometimes regexes are the best way to solve a problem, but you > really are > best served by trying other options first. In many cases, people > want to > use regexes when they should be using a full-blown parser because > regexes > require less up-front investment. Just remember that you'll get into > technical debt and pay more later. > > http://en.wikipedia.org/wiki/Technical_debt > -- > Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ > > "Many customs in this life persist because they ease friction and > promote > productivity as a result of universal agreement, and whether they are > precisely the optimal choices is much less important." --Henry Spencer > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies From motoom at xs4all.nl Wed Feb 24 16:20:15 2010 From: motoom at xs4all.nl (Michiel Overtoom) Date: Wed, 24 Feb 2010 16:20:15 +0100 Subject: [Baypiggies] More dramatic material? In-Reply-To: <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> Message-ID: <4B8543AF.30801@xs4all.nl> Glen Jarvis wrote: > I agree. Regex's have been my weakness forever because it just isn't > readable like python is readable. They just feel so unpythonic. Have a look at this Perl regexp, which validates an email address, but take care not to get nightmares from it ;-) http://www.ex-parrot.com/~pdw/Mail-RFC822-Address.html Greetings, From aahz at pythoncraft.com Wed Feb 24 16:36:35 2010 From: aahz at pythoncraft.com (Aahz) Date: Wed, 24 Feb 2010 07:36:35 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> Message-ID: <20100224153635.GA315@panix.com> On Wed, Feb 24, 2010, Glen Jarvis wrote: > > I'm always trying to fill in the fundamentals and I've been learning to > do fairly sophisticated regex's the past six months... However, my > fundamental question of 'why use it on Python' has been echoing in my > mind more and more with this conversation. Richard Pixley gave a few > reasons other than performance. Ah. Well, here are a couple of regexes that I've used recently in vim (the first one relies on some vim extensions to vi): :s/(\=\([0-9][0-9][0-9]\))\=[ ./]*\([0-9][0-9][0-9]\)[ .\-]*\([0-9][0-9][0-9][0-9]\)/\1-\2-\3/ /COMPANY_NAME.*folder snap (The first one is a regex I keep in my .exrc; the second one I typed in on the fly to search a log file. No-prize to the first person who figures out what the first regex does.) Generally speaking, regexes are good when you want to search for two things at the same time or to use the result of a search in a replace operation. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer From DennisR at dair.com Wed Feb 24 17:02:07 2010 From: DennisR at dair.com (Dennis Reinhardt) Date: Wed, 24 Feb 2010 08:02:07 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> Message-ID: <20100224160227.C9E1DF60A@mail.python.org> At 07:04 AM 2/24/2010, Glen Jarvis wrote: >Regex's have been my weakness forever because it just isn't >readable like python is readable. They just feel so unpythonic. In all my reading about RE, I have never seen (or remembered) a discussion of how to create a complex RE. They are always shown full-blown as if the purpose is to intimidate and imnpress. See http://www.texttoolkit.com/index.php?option=com_content&view=article&id=63:parsing-balanced-parenthesis-with-regular-expressions&catid=35:technology&Itemid=55 for a discussion of how a complex RE is constructed out of several smaller pieces. This method is particularly applicable if the parsing goal can be defined by a grammar (e.g. BNF) because you are handed a template for what the pieces are. --------------------------------- | Dennis | DennisR at dair.com | | Reinhardt | http://www.dair.com | --------------------------------- From aahz at pythoncraft.com Wed Feb 24 17:33:29 2010 From: aahz at pythoncraft.com (Aahz) Date: Wed, 24 Feb 2010 08:33:29 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <20100224160227.C9E1DF60A@mail.python.org> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <20100224160227.C9E1DF60A@mail.python.org> Message-ID: <20100224163329.GA14901@panix.com> One more thing: if you actually care about learning regex, you MUST purchase a copy of Friedl's _Mastering Regular Expressions_. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer From janssen at parc.com Wed Feb 24 17:47:25 2010 From: janssen at parc.com (Bill Janssen) Date: Wed, 24 Feb 2010 08:47:25 PST Subject: [Baypiggies] More dramatic material? In-Reply-To: <20100224153635.GA315@panix.com> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <20100224153635.GA315@panix.com> Message-ID: <64511.1267030045@parc.com> Just for fun, I grepped the UpLib Python sources for re.compile, and here's what I found. I usually use them to pull a field or fields out of a complicated expression. I'm particularly fond of the twisted and almost certainly broken-in-some-way URLPATTERN in plibUtil.py. Those interested in more aggressive use of regular expressions, particularly for natural language work, might want to look at this book: http://www.stanford.edu/~laurik/fsmbook/home.html or this paper http://www.cs.cmu.edu/afs/cs/user/alavie/11-711/711-cmt/Class-notes/lauri-et-al.pdf This software is available as a Python module at http://www.stanford.edu/~laurik/.book2software Bill ------------------------------------------------------------------------ addDocument.py:newline_expression = re.compile(r'(\s*\n)+') addDocument.py:whitespace_expression = re.compile(r'\s+') addDocument.py:charset_pattern = re.compile(r"^Content-Type:\s*text/plain;\s*charset=([^)]*)\n", re.IGNORECASE) addDocument.py:IMAGE_EXTENSIONS = re.compile(r'((\.gif)|(\.png)|(\.jpeg)|(\.jpg)|(\.tif)|(\.tiff)|(\.pbm)|(\.pnm)|(\.pgm)|(\.ppm)|(\.bmp)|(\.eps)|(\.rast)|(\.xbm)|(\.xpm))$', re.IGNORECASE) addDocument.py:TIFF_EXTENSION = re.compile(r'((\.tiff)|(\.tif))$', re.IGNORECASE) addDocument.py:PDF_EXTENSION = re.compile(r'\.pdf', re.IGNORECASE) addDocument.py:POSTSCRIPT_EXTENSION = re.compile(r'\.ps', re.IGNORECASE) addDocument.py:HTML_EXTENSION = re.compile(r'((\.html)|(\.htm))$', re.IGNORECASE) addDocument.py:MS_EXTENSIONS = re.compile(r'((\.ppt)|(\.pps)|(\.doc)|(\.dot)|(\.xls)|(\.xlt)|(\.docx)|(\.pptx)|(\.ppsx)|(\.xlsx))$', re.IGNORECASE) addDocument.py:FILE_TEXT_ENDING = re.compile(r'\stext$', re.IGNORECASE) addDocument.py:URL_PREFIX = re.compile(r'^((http:)|(https:)|(ftp:))') addDocument.py: SEPARATOR_PAGE_PATTERN = re.compile('(?:^|\f)([^\f]*UpLib Scan Separator Sheet){4}[^\f]*', re.MULTILINE) addDocument.py: FORMFEED_PATTERN = re.compile('\f') addDocument.py: TEXT_EXTENSION = re.compile(r'((\.txt)|(\.text))$', re.IGNORECASE) addDocument.py: CARD_SIZE_ENDING = re.compile(r'.*\.([0-9]+)x([0-9]+)$') angelHandler.py:CONTENT_LENGTH = re.compile ('Content-Length: ([0-9]+)', re.IGNORECASE) angelHandler.py:CONTENT_TYPE = re.compile ('Content-Type: (.+)', re.IGNORECASE) angelHandler.py:USER_AGENT = re.compile ('User-Agent: (.+)', re.IGNORECASE) angelHandler.py:HOST_HEADER = re.compile ('Host: (.+)', re.IGNORECASE) angelHandler.py:ACCEPTS_HEADER = re.compile ('Accept: (.+)', re.IGNORECASE) angelHandler.py:DOC_CHANGES_SINCE_HEADER = re.compile ('X-UpLib-Docs-Modified-Since: (.+)', re.IGNORECASE) angelHandler.py:TOUCHPATH = re.compile('/docs/(?P[^/]+)/(.*\.(html|pdf|tiff)|contents.txt)') angelHandler.py:DOCPATH = re.compile('/docs/(?P[^/]+)/.*\.(html|pdf|tiff|txt|png|bboxes)') angelHandler.py:COOKIE = re.compile('Cookie: (.+)', re.IGNORECASE) angelHandler.py:PASSWORD = re.compile('Password: (.+)', re.IGNORECASE) basicPlugins.py:SENSIBLE_BROWSERS = re.compile("Safari|Camino|Firefox|Konqueror|Gecko/|Opera") basicPlugins.py:SIMPLE_PAGE_NUMBER = re.compile(r"^-?[0-9]+$") basicPlugins.py:PAGE_RANGE = re.compile(r"^([0-9]+)(-|--)([0-9]+)$") basicPlugins.py:PAGE_NUMBERS = re.compile(r"^((b,[0-9]*,[0-9]+(-[0-9]+)*)|(d,[0-9]+,[0-9]+(-[0-9]+)*)|(r,[0-9]+,[0-9]+(-[0-9]+)*))(;" categories.py: re.compile(x.strip()) for x in configurator.default_configurator().get("categorize-excluded-categories", "").split(",") if x.strip()]} createHotSpots.py:URLPATTERN = re.compile(r'(^|\W)(?Phttp|ftp|https)://(?P[^/\s:]+)(:(?P[0-9]+)){0,1}(?P(/[-+~&$/\w,.?=%#]*[-+~$/\w?=%#])|\s*)') createHotSpots.py:CHARSET_PATTERN = re.compile(r"^Content-Type:\s*text/plain;\s*charset=([^)]*)\n", re.IGNORECASE) createPageBboxes.py:CHARSETPATTERN = re.compile(r"^Content-Type:\s*text/plain;\s*charset=([^)]*)\n", re.IGNORECASE) createPageBboxes.py: FRAGMENT_PATTERN = re.compile(r"^\|([0-9]+)\|(0|1)\|(0|1)\|[01]\|(0|1)\|([-A-Za-z0-9]+)\|([0-9]+)\|(.+)$") createThumbnails.py:PAGE_RANGE = re.compile(r"^([0-9]+)(-|--)([0-9]+)$") createThumbnails.py:PAGE_NUMBERS = re.compile(r"^((b,[0-9]*,[0-9]+(-[0-9]+)*)|(d,[0-9]+,[0-9]+(-[0-9]+)*)|(r,[0-9]+,[0-9]+(-[0-9]+)*))(;" document.py:CHARSET_PATTERN = re.compile(r"^Content-Type:\s*text/plain;\s*charset=([^)]*)\n", re.IGNORECASE) document.py:LANGUAGE_PATTERN = re.compile(r"^Content-Language:\s*(.*)\n", re.IGNORECASE) document.py:NEWLINE_PATTERN = re.compile(r"([^\r])\n", re.MULTILINE) emailParser.py:SUBJ_CRUFT_PATTERN = re.compile(r"^((?P[rR][eE]:)\s+)?\[[^]]+\]\s(?P.*)$") emailParser.py:HEADER_NEWLINE_PATTERN = re.compile("\n\s+", (re.MULTILINE|re.DOTALL)) extensions.py:CODE_EXCLUDES = re.compile(r"(.*~$)|(.*\.class$)|(.*\.pyc$)") indexing.py:PAGEBREAK = re.compile('\n\f\\r?\n') indexing.py: splitter = re.compile(name[p+1:]) pageview.py:CONTENT_TYPE = re.compile ('Content-Type: (.+)', re.IGNORECASE) paragraphs.py:REFERENCE_INDICATOR = re.compile('(\[[0-9]+\])|([0-9]+\.)|(\[[A-Za-z0-9]+\])') paragraphs.py:BULLET_INDICATOR = re.compile(u'\u2022\s+') paragraphs.py:IMAGE_TYPES = re.compile('(image/.*)|(application/vnd.ms-powerpoint)') plibUtil.py:URLPATTERN = re.compile(r'(^|\W)(?P(?Phttp|ftp|https)://(?P[-a-z0-9A-Z_\.]+[a-zA-Z0-9])(:(?P[0-9]+)){0,1}(?P(/[-+~&$/\w,.?=%#]*[-+~$/\w?=%#])|))') plibUtil.py:EMAILPATTERN = re.compile(r'(?P(?P[-a-zA-Z0-9._]+)@(?P[-a-zA-Z0-9_$.]+\.((com)|(edu)|(net)|(info)|(org)|(name)|([A-Za-z]{2})){1}))') plibUtil.py:GEMAILPATTERN = re.compile(r'({(?P([-a-zA-Z0-9._]+(\s*,\s*[-a-zA-Z0-9._]+)+))}@(?P[-a-zA-Z0-9_$.]+\.(com|edu|net|info|org|name|[A-Za-z]{2}){1}))') plibUtil.py:NOTELINE_BEGINNING = re.compile(r'(?P\A|\n)(?P.)', re.MULTILINE) plibUtil.py:DOC_ID_RE = re.compile('[0-9]{5}-[0-9]{2}-[0-9]{4}-[0-9]{3}') plibUtil.py:COLL_ID_RE = re.compile('[0-9]{5}-[0-9]{2}-[0-9]{4}-[0-9]{3}-[0-9]+') plibUtil.py:CHARSET_PATTERN = re.compile(r"^Content-Type:\s*text/plain;\s*charset=([^)]*)\n", re.IGNORECASE) plibUtil.py:LANGUAGE_PATTERN = re.compile(r"^Content-Language:\s*(.*)\n", re.IGNORECASE) plibUtil.py:PAGENUMBER = re.compile(r'page(\d+)\.png') plibUtil.py:UTF_8_EXPR = re.compile(r'^=\?UTF-8\?([BQbq])\?(.*)\?=$') plibUtil.py:CHARSETPATTERN = re.compile(r"^Content-Type:\s*text/plain;\s*charset=([^)]*)\n", re.IGNORECASE) ripper.py:newline_expression = re.compile(r'(\s*\n)+') ripper.py:whitespace_expression = re.compile(r'\s+') ripper.py:charset_pattern = re.compile(r"^Content-Type:\s*text/plain;\s*charset=([^)]*)\n", re.IGNORECASE) roman.py:romanNumeralPattern = re.compile(''' startAngel.py: update_directory_tree(dist_docfiles_dir, docroot, re.compile(r'.*\.pdf$|.*\.html$|.*\.css$|.*\.png$|.*\.js$|.*\.txt$')) webutils.py:_HTMLCHARREFPATTERN = re.compile("&#?[A-Z0-9a-z]+;") webutils.py:LOCATION_RE = re.compile('((?P[^@]+)@)?(?P[-A-Z._a-z0-9]+)(:(?P[0-9]+))?') webutils.py: COOKIE_LINE = re.compile("([^\s]+)\s+(TRUE|FALSE)\s+([^\s]+)\s+(TRUE|FALSE)\s+([0-9]+)\s+([^\s]+)\s+(.*)$") webutils.py: DATA_URI_PATTERN = re.compile(r'data:(?P[^/;]+?)/(?P[^/;]+?)(?P;[^;]+?)*?(;(?Pbase64))?,(?P.*)$', re.MULTILINE | re.DOTALL) webutils.py: CDATA_EXPR = re.compile(r'(/\*\<\!\[CDATA\[\*/)|(/\*\]\]\>\*/)|(//\<\!\[CDATA\[)|(//\]\]\>)') webutils.py: CSS_URL = re.compile(r"\s+url\(['\"]?([^)'\"]+)['\"]?\)") webutils.py: CSS_IMPORT = re.compile(r'(@import\s+"(?P[^"]+)"\s*;)|(@import\s+url\((?P[^)]+)\)\s*;)', re.MULTILINE) webutils.py: CDATA_EXPR = re.compile(r'(/\*\<\!\[CDATA\[\*/)|(/\*\]\]\>\*/)|(//\<\!\[CDATA\[)|(//\]\]\>)') xdocParser.py:MARKUP = re.compile(r"\[(([XYRa-z];[^\[\]]+\])|([A-Z]))", re.MULTILINE) xdocParser.py:FONT = re.compile(r"\[f;([0-9]+);\"([^\"]+)\";([A-Z]);([a-z]);[0-9]+;(F|V);[0-9]+;[0-9]+;[0-9]+;([0-9]+);[0-9]+\]") xdocParser.py:PAGE = re.compile(r"\[p;([0-9]+);(P|L);[0-9]+;[A-Z]+;([-0-9\.]+);([-0-9\.]+);([-0-9]+);([-0-9]+);([-0-9]+);([-0-9]+);[-0-9]+;[-0-9]+\]") xdocParser.py:WBOX = re.compile(r"\[b;([0-9]+);([0-9]+);([0-9]+);([0-9]+);([0-9]+);([0-9]+)\]") xdocParser.py:REGION = re.compile(r"\[t;([0-9]+);[0-9]+;([0-9]+);([0-9]+);[A-Z];[\"A-Za-z0-9]+;[\"A-Za-z0-9]+;[\"A-Za-z0-9]+;([0-9]+);([0-9]+);([0-9]+);([0-9]+);[0-9]+;[0-9]+\]") xdocParser.py:LANGUAGE = re.compile(r"\[O;([0-9]+);([0-9]+)\]") From jim at well.com Wed Feb 24 17:57:28 2010 From: jim at well.com (jim) Date: Wed, 24 Feb 2010 08:57:28 -0800 Subject: [Baypiggies] BayPIGgies meeting Thursday, February 25, 2010: Third-party RESTful Web Services Message-ID: <1267030648.7747.106.camel@jim-laptop> BayPIGgies meeting Thursday, February 25, 2010: Third-party RESTful Web Services Tonight's talk is * Techniques for Building Third-party RESTful Web Services by Shannon JJ Behrens Meetings usually start with a Newbie Nugget, a short discussion of an essential Python feature, especially for those new to Python. Tonight's Newbie Nugget: Python Code Quality by Simeon Franklin LOCATION Symantec Corporation Symantec Vcafe 350 Ellis Street Mountain View, CA 94043 http://maps.google.com/maps/ms?oe=utf-8&client=firefox-a&ie=UTF8&fb=1&split=1&gl=us&ei=w6i_Sfr6MZmQsQOzlv0v&hl=en&t=h&msa=0&msid=116202735295394761637.00046550c09ff3d96bff1&ll=37.397693,-122.053707&spn=0.002902,0.004828&z=18 BayPIGgies meeting information is available at http://www.baypiggies.net/ ------------------------ Agenda ------------------------ ..... 7:30 PM ........................... General hubbub, inventory end-of-meeting announcements, any first-minute announcements. ..... 7:35 PM to 7:45 PM ................ Newbie Nugget: Python Code Quality by Simeon Franklin ..... 7:45 PM to 8:40 PM (or so) ................ * Techniques for Building Third-party RESTful Web Services by Shannon JJ Behrens Abstract: Aside from covering a lot of material from the book "RESTful Web Services", this talk presents a handful of tips and tricks for creating a third-party RESTful Web Service meant for consumption by Web browsers. ..... 8:30 PM to 9:20 PM ................ Mapping and Random Access Mapping is a rapid-fire audience announcement of issues, hiring, events, and other topics. Random Access follows people immediately to allow follow up on the announcements and other interests. From rich at noir.com Wed Feb 24 18:13:45 2010 From: rich at noir.com (K. Richard Pixley) Date: Wed, 24 Feb 2010 09:13:45 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <20100224153635.GA315@panix.com> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <20100224153635.GA315@panix.com> Message-ID: <4B855E49.1090002@noir.com> I concur with Aahz's reasons for using regular expressions. I also concur that they tend to be write only. However, witty quote aside, I don't agree about them being distinct from a "full-blown parser" - more like "full-blown parser lite". I did some work in ICON, (http://en.wikipedia.org/wiki/Icon_(programming_language)), for a while, which is a fascinating language. It includes goal oriented expressions, which have a feel much like functional programming languages like ML, PROLOG, or Mercury; innate string handling like it's predecessor, snobol, (both are by Ralph Griswold); and generators, which are built almost identically to python's 'yield' statement and essentially create closures, a la CLU. All this over 30 years ago in 1977. In any case, the goal oriented expressions combined with string handling functions of ICON are a lot like writing regular expressions, with side effects. And the side effects tend to Do The Right Thing, which is more direct and more useful than the repeating terminology of regular expressions. From my perspective, the biggest reason to use regular expressions is when you need to pull out the matched portion in order to further process it, (as Aahz mentioned, and hence my repeating examples). But IMO, this /is/ a form of parser generation, although I'll admit that it's extremely dense code that tends to be unfamiliar to most programmers. (But, then, it's possible to write some /very/ dense code in python too). I keep looking for an opportunity to learn ANTLR, (http://www.antlr.org), which is capable of generating parsers in python, but the majority of problems I run into can be solved with a "reduction in strength" of the tools involved. I can't imagine that the learning curve for antlr is much less than the learning curve for regular expressions. I haven't even read through the regular expression library in python and frankly, I probably never will. If it comes to that, I'll probably use other tools. I /do/ use regular expressions in sed fairly regularly. --rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at zachary.com Wed Feb 24 18:41:23 2010 From: david at zachary.com (David Creemer) Date: Wed, 24 Feb 2010 09:41:23 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <4B8543AF.30801@xs4all.nl> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <4B8543AF.30801@xs4all.nl> Message-ID: FYI- If you are really interested in regular expression performance, I suggest reading this short paper: http://swtch.com/~rsc/regexp/regexp1.html There is an old feature request bug to get this into the Python library: http://bugs.python.org/issue1662581 There are various implementations floating around the net if you search... -- David From cappy2112 at gmail.com Wed Feb 24 20:15:58 2010 From: cappy2112 at gmail.com (Tony Cappellini) Date: Wed, 24 Feb 2010 11:15:58 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <20100224163329.GA14901@panix.com> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <20100224160227.C9E1DF60A@mail.python.org> <20100224163329.GA14901@panix.com> Message-ID: <8249c4ac1002241115k11eb952fi4b0d43e1f6847c0f@mail.gmail.com> > > >>One more thing: if you actually care about learning regex, you MUST > >>purchase a copy of Friedl's _Mastering Regular Expressions_. > +100 If you're a casual user of regular expressions (1-2 times/year) , as I am, and don't use them frequently enough to learn them well, this book may help you out in a pinch http://www.baypiggies.net/index_html/user-group-association-program/BookReviews/regular-expressions-cookbook A good regex debugger tool is also invaluable, so you can generate/test your regular expression in a stand-alone environment. (so you don't wait until your program is at the point of execution where the regex is invoked) I've used this one quite a bit, and it supports ~16 regular expression flavors http://www.regexbuddy.com/ Kodos is a regex debugger written in qt/Python, but I've had too many problems with it and stopped using it. It's not actively supported any more. -------------- next part -------------- An HTML attachment was scrubbed... URL: From knupp at well.com Wed Feb 24 21:49:07 2010 From: knupp at well.com (David Knupp) Date: Wed, 24 Feb 2010 12:49:07 -0800 (PST) Subject: [Baypiggies] Move over, Unladen Swallow... (humor) In-Reply-To: <8249c4ac1002241115k11eb952fi4b0d43e1f6847c0f@mail.gmail.com> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <20100224160227.C9E1DF60A@mail.python.org> <20100224163329.GA14901@panix.com> <8249c4ac1002241115k11eb952fi4b0d43e1f6847c0f@mail.gmail.com> Message-ID: ...and make room for, Like, Python... http://www.staringispolite.com/likepython/ * Apologies if this has crossed the list before -- I don't always keep up with every post. From mvoorhie at yahoo.com Wed Feb 24 22:09:41 2010 From: mvoorhie at yahoo.com (Mark Voorhies) Date: Wed, 24 Feb 2010 13:09:41 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <20100224153635.GA315@panix.com> References: <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <20100224153635.GA315@panix.com> Message-ID: <201002241309.41605.mvoorhie@yahoo.com> On Wednesday 24 February 2010 7:36 am Aahz wrote: > Generally speaking, regexes are good when you want to search for two > things at the same time or to use the result of a search in a replace > operation. A nice Python feature for this case is being able to assign names to the matched groups, e.g.: import re locus_re = re.compile( "(^(?P[^:]+)\:(?P[\d]+)\.\.(?P[\d]+)$)", re.M) class Locus: def __init__(self, ref, start, stop, strand = "+"): self.ref = str(ref) self.start = int(start) self.stop = int(stop) open("loci.txt","w").write("\n".join(( "contig1:1..100", "contig2:1..50", "contig3:75..600"))+"\n") loci = [Locus(strand = "-", **(i.groupdict())) for i in locus_re.finditer(open("loci.txt").read())] --Mark From aahz at pythoncraft.com Wed Feb 24 22:53:51 2010 From: aahz at pythoncraft.com (Aahz) Date: Wed, 24 Feb 2010 13:53:51 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <201002241309.41605.mvoorhie@yahoo.com> References: <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <20100224153635.GA315@panix.com> <201002241309.41605.mvoorhie@yahoo.com> Message-ID: <20100224215351.GA15158@panix.com> On Wed, Feb 24, 2010, Mark Voorhies wrote: > > locus_re = re.compile( > "(^(?P[^:]+)\:(?P[\d]+)\.\.(?P[\d]+)$)", re.M) Reminder: always use raw strings with regexes ;-) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer From dalke at dalkescientific.com Thu Feb 25 00:03:00 2010 From: dalke at dalkescientific.com (Andrew Dalke) Date: Wed, 24 Feb 2010 18:03:00 -0500 Subject: [Baypiggies] More dramatic material? In-Reply-To: References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <4B8543AF.30801@xs4all.nl> Message-ID: <42AA53B2-F039-45FF-95B1-D43CFF6AFA81@dalkescientific.com> On Feb 24, 2010, at 12:41 PM, David Creemer wrote: > There is an old feature request bug to get this into the Python library: > > http://bugs.python.org/issue1662581 While it's a request, the pain is mostly theoretical and arises in three main cases: - regular expressions written by someone who have no clue about what's going on (image writing regular expressions to pattern match entire tag groups in a multi-MB text field). These are often best solved through some other means, like rewriting the regexp so it doesn't need any backtracking. - people trying to push regular expressions to do bizarre things (like match only against input strings of length which is not a prime). These don't occur in real life. - people worried about denial-of-security attack through user-defined regexps. They want to remove regexp support entirely. The solution (using the Thompson algorithm) has some costs. It does not allow back-references and it does not allow grouping. Consider the pattern for the algorithm mentioned in the second of these (match a string of 1:s of non-prime length): /^1?$|^(11+?)\1+$/ The Thompson algorithm cannot implement it, because the Thompson algorithm only supports "true" (in the computer science sense) regular expressions, and those cannot do primality testing. It's possible to detect input patterns which don't need grouping and back-references, but that's extra implementation overhead, and the result would not change these three cases. Andrew dalke at dalkescientific.com From knupp at well.com Thu Feb 25 02:03:36 2010 From: knupp at well.com (David Knupp) Date: Wed, 24 Feb 2010 17:03:36 -0800 (PST) Subject: [Baypiggies] Move over, Unladen Swallow... (humor) In-Reply-To: <6FD4EF64-7295-4F83-8002-DB2EE07BF79D@me.com> References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <20100224160227.C9E1DF60A@mail.python.org> <20100224163329.GA14901@panix.com> <8249c4ac1002241115k11eb952fi4b0d43e1f6847c0f@mail.gmail.com> <6FD4EF64-7295-4F83-8002-DB2EE07BF79D@me.com> Message-ID: I sense a potential code competition. On Wed, 24 Feb 2010, Grace Law wrote: > lol... David, you beat me to it. I was just going to post this. > > Like, Python is a "just for fun" project started by Jonathan Howard, one of > our engineers at Lolapps, after a night of drinking in celebration of hiring > another cool engineer. Enjoy! :-) > > Grace > > On Feb 24, 2010, at 12:49 PM, David Knupp wrote: > >> ...and make room for, Like, Python... >> http://www.staringispolite.com/likepython/ >> >> * Apologies if this has crossed the list before -- I don't always keep up >> with every post. >> >> >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> http://mail.python.org/mailman/listinfo/baypiggies > From aleksandr.goretoy at gmail.com Thu Feb 25 02:16:54 2010 From: aleksandr.goretoy at gmail.com (alex goretoy) Date: Wed, 24 Feb 2010 19:16:54 -0600 Subject: [Baypiggies] Move over, Unladen Swallow... (humor) In-Reply-To: References: <3cce76f61002240347pebf3059j19c9648af73302ba@mail.gmail.com> <20100224143818.GA23116@panix.com> <54C148A2-BB2C-4412-B491-3E7A614AF696@glenjarvis.com> <20100224160227.C9E1DF60A@mail.python.org> <20100224163329.GA14901@panix.com> <8249c4ac1002241115k11eb952fi4b0d43e1f6847c0f@mail.gmail.com> <6FD4EF64-7295-4F83-8002-DB2EE07BF79D@me.com> Message-ID: lol, dude like python is awesome, hahaha what a great idea -Alex Goretoy -------------- next part -------------- An HTML attachment was scrubbed... URL: From dalke at dalkescientific.com Thu Feb 25 04:28:03 2010 From: dalke at dalkescientific.com (Andrew Dalke) Date: Wed, 24 Feb 2010 22:28:03 -0500 Subject: [Baypiggies] More dramatic material? In-Reply-To: References: Message-ID: <22EF4345-3F6F-48BC-AA51-7F03E7AA67EE@dalkescientific.com> On Feb 23, 2010, at 9:23 PM, Glen Jarvis wrote: > total = 0 > for line in f.readlines(): > total = total + line.upper().count("THE") I don't see that anyone else mentioned this - your code here also matches "there" and "other" and quite a few other words. You can't easily use count to count the number of "the" words with this approach, since it's hard to include lines like late Emperor, and was nicknamed 'the King of Prussia.' He is very clever and "'The Brook,'" suggested Nicholas. but exclude lines like "There, leave Bruin alone; here's a bet on." and bathe. Regular expressions make this rather easy to do, since you can define a pattern like "(^|\W)the(\W|$)", re.I to match "the" which is either at the start of the line or has a non-word character, followed by "the", followed by a non-word character or the end of line. Plus, performance-wise it's best to work with a block of text at a time, and not a line at a time. You can get some of the speed advantage by using the default iterator for a file instead of 'readline' -- the former reads a block of text at a time and breaks up the newlines manually, while the latter reads a character at a time, which takes a lot of system calls. That is, try timing it with: total = 0 for line in f: total = total + line.upper().count("THE") Or, try using: total = 0 while 1: block = f.read(10000) + f.readline() if not block: break total = total + block.upper().count("THE") [Using a FASTA file] > Again, repetition gives very similar numbers. Using good old pure python (removing the 'upper()': > real 0m5.268s > user 0m4.474s > sys 0m0.715s > > Using the equivalent of a fast regular expression (no special matching needed): > > m = re.split(r'TGGCCC', contents) > > we get the same results and in time: > > real 0m5.118s > user 0m2.702s > sys 0m1.214s > > Now, we're looking at a large increase. But, again, a factor of about two or less? Is that repeatable and consistent? I'm surprised at those numbers, since both implementations should essentially be the same. That is, I think the re module looks at the input string, realizes its a constant string, and uses the same algorithm as string.find. > Can you think of a better example than this? Something more 'wow'.. Staying in bioinformatics, try parsing fields from a BLAST output file. For something more general, look in the Python standard library for instances of re.compile. Here's a definition from decimal.py: _parser = re.compile(r""" # A numeric string consists of: # \s* (?P[-+])? # an optional sign, followed by either... ( (?=\d|\.\d) # ...a number (with at least one digit) (?P\d*) # having a (possibly empty) integer part (\.(?P\d*))? # followed by an optional fractional part (E(?P[-+]?\d+))? # followed by an optional exponent, or... | Inf(inity)? # ...an infinity, or... | (?Ps)? # ...an (optionally signaling) NaN # NaN (?P\d*) # with (possibly empty) diagnostic info. ) # \s* \Z """, re.VERBOSE | re.IGNORECASE | re.UNICODE).match Try doing that without a regular expression. Andrew dalke at dalkescientific.com From glen at glenjarvis.com Thu Feb 25 05:35:56 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Wed, 24 Feb 2010 20:35:56 -0800 Subject: [Baypiggies] More dramatic material? In-Reply-To: <22EF4345-3F6F-48BC-AA51-7F03E7AA67EE@dalkescientific.com> References: <22EF4345-3F6F-48BC-AA51-7F03E7AA67EE@dalkescientific.com> Message-ID: But so did the equivalent regular expression, yes? I compared that the count gave the same results in both. But, you're right. My example is poor since I said I was counting the word 'the'. As I'm sure it was obvious to you, the DNA example wasn't a real life example either :( Who searches DNA for exact matches only instead of BLASTing? I dunno.. maybe there's a reason for it. But, the best example for the medium... Cheers, Glen On Wed, Feb 24, 2010 at 7:28 PM, Andrew Dalke wrote: > On Feb 23, 2010, at 9:23 PM, Glen Jarvis wrote: > > total = 0 > > for line in f.readlines(): > > total = total + line.upper().count("THE") > > I don't see that anyone else mentioned this - your code here also matches > "there" and "other" and quite a few other words. You can't easily use count > to count the number of "the" words with this approach, since it's hard to > include lines like > > late Emperor, and was nicknamed 'the King of Prussia.' He is very clever > and > "'The Brook,'" suggested Nicholas. > > but exclude lines like > > "There, leave Bruin alone; here's a bet on." > and > bathe. > > > Regular expressions make this rather easy to do, since you can define a > pattern like > > "(^|\W)the(\W|$)", re.I > > to match "the" which is either at the start of the line or has a non-word > character, followed by "the", followed by a non-word character or the end of > line. > > > > Plus, performance-wise it's best to work with a block of text at a time, > and not a line at a time. You can get some of the speed advantage by using > the default iterator for a file instead of 'readline' -- the former reads a > block of text at a time and breaks up the newlines manually, while the > latter reads a character at a time, which takes a lot of system calls. > > That is, try timing it with: > > total = 0 > for line in f: > total = total + line.upper().count("THE") > > Or, try using: > > total = 0 > while 1: > block = f.read(10000) + f.readline() > if not block: > break > total = total + block.upper().count("THE") > > > [Using a FASTA file] > > Again, repetition gives very similar numbers. Using good old pure python > (removing the 'upper()': > > real 0m5.268s > > user 0m4.474s > > sys 0m0.715s > > > > Using the equivalent of a fast regular expression (no special matching > needed): > > > > m = re.split(r'TGGCCC', contents) > > > > we get the same results and in time: > > > > real 0m5.118s > > user 0m2.702s > > sys 0m1.214s > > > > Now, we're looking at a large increase. But, again, a factor of about two > or less? > > Is that repeatable and consistent? I'm surprised at those numbers, since > both implementations should essentially be the same. That is, I think the re > module looks at the input string, realizes its a constant string, and uses > the same algorithm as string.find. > > > Can you think of a better example than this? Something more 'wow'.. > > Staying in bioinformatics, try parsing fields from a BLAST output file. For > something more general, look in the Python standard library for instances of > re.compile. Here's a definition from decimal.py: > > _parser = re.compile(r""" # A numeric string consists of: > # \s* > (?P[-+])? # an optional sign, followed by either... > ( > (?=\d|\.\d) # ...a number (with at least one digit) > (?P\d*) # having a (possibly empty) integer part > (\.(?P\d*))? # followed by an optional fractional part > (E(?P[-+]?\d+))? # followed by an optional exponent, or... > | > Inf(inity)? # ...an infinity, or... > | > (?Ps)? # ...an (optionally signaling) > NaN # NaN > (?P\d*) # with (possibly empty) diagnostic info. > ) > # \s* > \Z > """, re.VERBOSE | re.IGNORECASE | re.UNICODE).match > > > Try doing that without a regular expression. > > > Andrew > dalke at dalkescientific.com > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cappy2112 at gmail.com Thu Feb 25 06:54:22 2010 From: cappy2112 at gmail.com (Tony Cappellini) Date: Wed, 24 Feb 2010 21:54:22 -0800 Subject: [Baypiggies] Python Magazine needs a new editor Message-ID: <8249c4ac1002242154r63fc7e34j5f5f7727be714e9c@mail.gmail.com> Following up with my previous email about Python Magazine loosing their editor .... There are many gifted people in Baypiggies who may be interested in helping out Python Magazine. Interested parties should inquire at the contact information below.. "We are sorry to see Brandon go. We are looking for a new editor and plan to continue with the magazine. Thanks, Cathleen" -- cathleen at phparch.com or info at phparch.com php|architect and Python Magazine Marco Tabini and Associates 28 Bombay Avenue Toronto, Ontario M3H 1B7 Canada Telephone: 416 630-6202 ext 224 or 1 877 630-6202 ext 224 North America only Fax: (416) 630-5057 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dalke at dalkescientific.com Thu Feb 25 11:54:35 2010 From: dalke at dalkescientific.com (Andrew Dalke) Date: Thu, 25 Feb 2010 05:54:35 -0500 Subject: [Baypiggies] More dramatic material? In-Reply-To: References: <22EF4345-3F6F-48BC-AA51-7F03E7AA67EE@dalkescientific.com> Message-ID: <1F69DE67-3A64-4AB8-90FB-58662753C2AD@dalkescientific.com> On Feb 24, 2010, at 11:35 PM, Glen Jarvis wrote: > But so did the equivalent regular expression, yes? I compared that the count gave the same results in both. Oh, absolutely. It's just that regular expressions have a clear path to being able to solve the problem while with the other Python string operations it's a lot trickier. One solution is to make a translation table >>> import string >>> to_list = [" ",]*256 >>> to_list[ord("A"):ord("Z")+1] = string.ascii_lowercase >>> to_list[ord("a"):ord("z")+1] = string.ascii_lowercase >>> table = string.maketrans( "".join(chr(i) for i in range(256)), "".join(to_list)) >>> line = "The best and the worst there ever was." >>> line.translate(table) 'the best and the worst there ever was ' >>> (" " + line.translate(table) + " ").count(" the ") 2 >>> vs. >>> re.findall(r"(^|\W)the(\W|$)", line, re.I) [('', ' '), (' ', ' ')] >>> len(re.findall(r"(^|\W)the(\W|$)", line, re.I)) 2 >>> Andrew dalke at dalkescientific.com From spmcinerney at hotmail.com Fri Feb 26 00:27:38 2010 From: spmcinerney at hotmail.com (Stephen McInerney) Date: Thu, 25 Feb 2010 15:27:38 -0800 Subject: [Baypiggies] Ride to EastBay/BART after BayPIGgies Message-ID: I can offer a one-way ride back to the EastBay/ Oakland/Alameda/Fremont/BART after BayPIGgies tonight 9:30pm. Contact me offlist if interested. Stephen _________________________________________________________________ Hotmail: Powerful Free email with security by Microsoft. http://clk.atdmt.com/GBL/go/201469230/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From simeonf at gmail.com Fri Feb 26 08:01:17 2010 From: simeonf at gmail.com (Simeon Franklin) Date: Thu, 25 Feb 2010 23:01:17 -0800 Subject: [Baypiggies] Tonight's Newbie Nugget on Code Quality Message-ID: I posted the text for my newbie nugget to my blog at http://simeonfranklin.com/blog/2010/feb/25/python-code-quality/ I also included links to Clonedigger and Pygenie and the entire 3.7 Mb clonedigger report on Django trunk if anybody is interested... -regards Simeon Franklin From glen at glenjarvis.com Fri Feb 26 16:47:32 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Fri, 26 Feb 2010 07:47:32 -0800 Subject: [Baypiggies] PyCon in SF Message-ID: We are planning a PyCon in SF get-together where we will watch Pycon Videos, have time for discussion, bring in some food and beverage, and socialize. My experience with planning a group SpamALot is that not everyone wants to be involved in making the decisions for time, location, place, etc. However, some people do. We are building a planning team and we already have three people on the 'committee.' If you wish to be involved in bringing a small slice of the PyCon experience to those of us who didn't get to go, please send me an email. We hope to plan this quickly, and the Videos are already available. So, please email me and let's all get together to make this happen! :) Cheers, Glen -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Fri Feb 26 17:20:02 2010 From: aahz at pythoncraft.com (Aahz) Date: Fri, 26 Feb 2010 08:20:02 -0800 Subject: [Baypiggies] PyCon in SF In-Reply-To: References: Message-ID: <20100226162002.GA19591@panix.com> On Fri, Feb 26, 2010, Glen Jarvis wrote: > > We are planning a PyCon in SF get-together where we will watch Pycon Videos, > have time for discussion, bring in some food and beverage, and socialize. > > My experience with planning a group SpamALot is that not everyone wants to > be involved in making the decisions for time, location, place, etc. However, > some people do. We are building a planning team and we already have three > people on the 'committee.' If you wish to be involved in bringing a small > slice of the PyCon experience to those of us who didn't get to go, please > send me an email. Just to be clear, I assume this is separate from the possible PyCon in SF 2012? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer From alecf at flett.org Fri Feb 26 17:51:43 2010 From: alecf at flett.org (Alec Flett) Date: Fri, 26 Feb 2010 08:51:43 -0800 Subject: [Baypiggies] PyCon in SF In-Reply-To: References: Message-ID: Might I suggest: http://whenisgood.net/ for figuring out a day/time.. Alec On Fri, Feb 26, 2010 at 7:47 AM, Glen Jarvis wrote: > We are planning a PyCon in SF get-together where we will watch Pycon > Videos, have time for discussion, bring in some food and beverage, and > socialize. > > My experience with planning a group SpamALot is that not everyone wants to > be involved in making the decisions for time, location, place, etc. However, > some people do. We are building a planning team and we already have three > people on the 'committee.' If you wish to be involved in bringing a small > slice of the PyCon experience to those of us who didn't get to go, please > send me an email. > > We hope to plan this quickly, and the Videos are already available. So, > please email me and let's all get together to make this happen! :) > > Cheers, > > > > Glen > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From glen at glenjarvis.com Fri Feb 26 18:05:45 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Fri, 26 Feb 2010 09:05:45 -0800 Subject: [Baypiggies] Dump of PyCon 2010 Was: PyCon in SF In-Reply-To: <20100226162002.GA19591@panix.com> References: <20100226162002.GA19591@panix.com> Message-ID: <0AFB6724-ECC9-4D8C-A6E6-2894DBE5D6CB@glenjarvis.com> Aaackk.... I chose a very poor subject name. Many of us couldn't go to PyCon and we want to grab a piece of the fun and knowledge. With Jims help, we have access to a space in SF that we can host a lot of us, project our favorite videos/talks from PyCon this year, socialize, maybe do a tutorial or two, and try our darndest to make it a mini reinactment of PyCon for a Saturday from around 11am to 9pm. Our list of committee members to help make this happen is now around six... Awesome. :) Glen On Feb 26, 2010, at 8:20 AM, Aahz wrote: > On Fri, Feb 26, 2010, Glen Jarvis wrote: >> >> We are planning a PyCon in SF get-together where we will watch >> Pycon Videos, >> have time for discussion, bring in some food and beverage, and >> socialize. >> >> My experience with planning a group SpamALot is that not everyone >> wants to >> be involved in making the decisions for time, location, place, etc. >> However, >> some people do. We are building a planning team and we already have >> three >> people on the 'committee.' If you wish to be involved in bringing a >> small >> slice of the PyCon experience to those of us who didn't get to go, >> please >> send me an email. > > Just to be clear, I assume this is separate from the possible PyCon > in SF > 2012? > -- > Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ > > "Many customs in this life persist because they ease friction and > promote > productivity as a result of universal agreement, and whether they are > precisely the optimal choices is much less important." --Henry Spencer > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies From laban.patrick at gmail.com Fri Feb 26 18:10:44 2010 From: laban.patrick at gmail.com (Patrick Laban) Date: Fri, 26 Feb 2010 09:10:44 -0800 Subject: [Baypiggies] PyCon in SF In-Reply-To: References: Message-ID: <5ee958ff1002260910w3a21ab3i870f89039940bec1@mail.gmail.com> Very interested but can't make plans right now. I'm getting a job offer and need to have that settled first. On Fri, Feb 26, 2010 at 7:47 AM, Glen Jarvis wrote: > >> We are planning a PyCon in SF get-together where we will watch Pycon >> Videos, have time for discussion, bring in some food and beverage, and >> socialize. >> >> My experience with planning a group SpamALot is that not everyone wants to >> be involved in making the decisions for time, location, place, etc. However, >> some people do. We are building a planning team and we already have three >> people on the 'committee.' If you wish to be involved in bringing a small >> slice of the PyCon experience to those of us who didn't get to go, please >> send me an email. >> >> We hope to plan this quickly, and the Videos are already available. So, >> please email me and let's all get together to make this happen! :) >> >> Cheers, >> >> >> >> Glen >> >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> http://mail.python.org/mailman/listinfo/baypiggies >> > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim at well.com Fri Feb 26 18:48:06 2010 From: jim at well.com (jim) Date: Fri, 26 Feb 2010 09:48:06 -0800 Subject: [Baypiggies] Dump of PyCon 2010 Was: PyCon in SF In-Reply-To: <0AFB6724-ECC9-4D8C-A6E6-2894DBE5D6CB@glenjarvis.com> References: <20100226162002.GA19591@panix.com> <0AFB6724-ECC9-4D8C-A6E6-2894DBE5D6CB@glenjarvis.com> Message-ID: <1267206486.6669.12.camel@jim-laptop> should we have an email group of the six so's to coordinate without adding noise to the baypiggies' list? or does it work that you, glen, are the pointman? the prospective location is noisebridge http://www.noisebridge.net on the third floor of 2169 mission street very near 18th street in san francisco, roughly 5000 square feet divided into various work areas and classrooms. knowing in advance how many persons will show up at what times on whatever saturday we end up with will help in planning, so if you know you're likely to attend, please let us know one way or another. jim On Fri, 2010-02-26 at 09:05 -0800, Glen Jarvis wrote: > Aaackk.... I chose a very poor subject name. Many of us couldn't go to > PyCon and we want to grab a piece of the fun and knowledge. > > With Jims help, we have access to a space in SF that we can host a lot > of us, project our favorite videos/talks from PyCon this year, > socialize, maybe do a tutorial or two, and try our darndest to make it > a mini reinactment of PyCon for a Saturday from around 11am to 9pm. > > Our list of committee members to help make this happen is now around > six... Awesome. :) > > Glen > > On Feb 26, 2010, at 8:20 AM, Aahz wrote: > > > On Fri, Feb 26, 2010, Glen Jarvis wrote: > >> > >> We are planning a PyCon in SF get-together where we will watch > >> Pycon Videos, > >> have time for discussion, bring in some food and beverage, and > >> socialize. > >> > >> My experience with planning a group SpamALot is that not everyone > >> wants to > >> be involved in making the decisions for time, location, place, etc. > >> However, > >> some people do. We are building a planning team and we already have > >> three > >> people on the 'committee.' If you wish to be involved in bringing a > >> small > >> slice of the PyCon experience to those of us who didn't get to go, > >> please > >> send me an email. > > > > Just to be clear, I assume this is separate from the possible PyCon > > in SF > > 2012? > > -- > > Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ > > > > "Many customs in this life persist because they ease friction and > > promote > > productivity as a result of universal agreement, and whether they are > > precisely the optimal choices is much less important." --Henry Spencer > > _______________________________________________ > > Baypiggies mailing list > > Baypiggies at python.org > > To change your subscription options or unsubscribe: > > http://mail.python.org/mailman/listinfo/baypiggies > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > http://mail.python.org/mailman/listinfo/baypiggies > From glen at glenjarvis.com Fri Feb 26 19:10:50 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Fri, 26 Feb 2010 10:10:50 -0800 Subject: [Baypiggies] Dump of PyCon 2010 Was: PyCon in SF In-Reply-To: <1267206486.6669.12.camel@jim-laptop> References: <20100226162002.GA19591@panix.com> <0AFB6724-ECC9-4D8C-A6E6-2894DBE5D6CB@glenjarvis.com> <1267206486.6669.12.camel@jim-laptop> Message-ID: On Fri, Feb 26, 2010 at 9:48 AM, jim wrote: > should we have an email group of the six > so's to coordinate without adding noise to > the baypiggies' list? or does it work that > you, glen, are the pointman? > I'm point-man. I'll make an email list/group by midnight tonight. I'll also put together some infrastructure so that we can better get a feel for who wishes to attend. I'll be out of communication for most of the day until about 6 pm, but don't mind doing what I can to make this a success. Although I'll automate the "who's attending", you are more than welcome to send me a personal email to let me know you're interested. I'll review all related emails and organize this in the next 48 hours (starting after 6 pm tonight). Cheers, Glen -------------- next part -------------- An HTML attachment was scrubbed... URL: From jjinux at gmail.com Sat Feb 27 04:15:07 2010 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri, 26 Feb 2010 19:15:07 -0800 Subject: [Baypiggies] Techniques for Building Third-party RESTful Web Services Message-ID: Thanks to everyone who came to my talk. Here are the slides: http://jjinux.blogspot.com/2009/06/talk-techniques-for-building-third.html Happy Hacking! -jj -- In this life we cannot do great things. We can only do small things with great love. -- Mother Teresa http://jjinux.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From larryt at winfirst.com Sun Feb 28 02:21:25 2010 From: larryt at winfirst.com (larryt at winfirst.com) Date: Sat, 27 Feb 2010 17:21:25 -0800 (PST) Subject: [Baypiggies] More dramatic material? In-Reply-To: <1F69DE67-3A64-4AB8-90FB-58662753C2AD@dalkescientific.com> References: <22EF4345-3F6F-48BC-AA51-7F03E7AA67EE@dalkescientific.com> <1F69DE67-3A64-4AB8-90FB-58662753C2AD@dalkescientific.com> Message-ID: <20100227172125.APD25823@ms1.mc.surewest.net> Hi All, Before this recent thread about regular expressions gets too cold, I'm looking for a python project whose goal was/is to mostly replace need to use re's. It showed up in some list or other a year or more ago, and I only had time to glance at it then. In that quick look it didn't appear to be a regular expression builder. I've tried to find it since, but without success. Does anybody recall something like that? thanks, larry From cappy2112 at gmail.com Sun Feb 28 03:40:23 2010 From: cappy2112 at gmail.com (Tony Cappellini) Date: Sat, 27 Feb 2010 18:40:23 -0800 Subject: [Baypiggies] Matplotlib for Python Developers Message-ID: <8249c4ac1002271840x30125430xdc0b7e9489148744@mail.gmail.com> Has anyone read this book? http://tinyurl.com/yc34pn9 If so- what are your thoughts on it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From glen at glenjarvis.com Sun Feb 28 04:41:43 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Sat, 27 Feb 2010 19:41:43 -0800 Subject: [Baypiggies] PyCon in SF In-Reply-To: References: Message-ID: Thank you for volunteering to help with organizing a sort of 'after pycon braindump.' [Actually, this is a form letter and I'm not certain you did volunteer. You have a great suggestion on picking a tool to pick a date ahead of time.] There are so many people in this area that are interested, I think we can get the same type of energy. I envision this to be a place for both openspaces (aka unconferences) and for reviewing our favorite videos. If you will, take a quick moment to see what the open space is about: http://us.pycon.org/2010/openspace/ I want to get the creative juices flowing, but it also helps to have a little infrastructure. Since it's only been one day, the emphasis will be on little. I have created a page of all of the videos that I have found. One can also vote anonymously on the talks you prefer on this page. There's no security, so someone could vote a million times. But, for this level, I think that's okay. That website is http://pycon.glenjarvis.com/ You will see that: * I chose a boring name (pycon and also pycon_dump in places in code). I need help finding a good name to use in all of these places, * The links on the page (except for "I like") aren't yet activated * I hadn't made any account login pages yet * More details will launch a new window under the subject area This was knocked up quickly with Django and I will soon put this in a repository so others can chip in, edit code, review code, etc. Also, the data is in the Admin and I've created an account just for you. Go to this URL: http://pycon.glenjarvis.com/admin/ Your username is: alecf Your password is: )BtjV^5Y You are currently super user -- you could wipe everything out -- or add a million records. Again, for this stage creative juices are more important to me than data security. There's nothing of consequence (except my hard work to scrape this data) that is in this database. Passwords are encrypted, but others organizing will also have super user access. So, don't pick a password that would compromise you anywhere -- just in case. And, finally, it is *MUCH* easier for me to use Google Wave than it is to manage so many email threads. I know some people have concerns about having a Google Account. For those people I will manually cut and paste messages to them. I really hope we can use Google Wave -- it will make my life so much easier. My Google Wave account is glen.r.jarvis and I've created a wave with title "Organizing a PostPyCon Meetup." Feel free to poke around as we start building this from the very very basics to something awesome? Cheers, Glen On Fri, Feb 26, 2010 at 8:51 AM, Alec Flett wrote: > Might I suggest: > > http://whenisgood.net/ > > for figuring out a day/time.. > > Alec > > On Fri, Feb 26, 2010 at 7:47 AM, Glen Jarvis wrote: > >> We are planning a PyCon in SF get-together where we will watch Pycon >> Videos, have time for discussion, bring in some food and beverage, and >> socialize. >> >> My experience with planning a group SpamALot is that not everyone wants to >> be involved in making the decisions for time, location, place, etc. However, >> some people do. We are building a planning team and we already have three >> people on the 'committee.' If you wish to be involved in bringing a small >> slice of the PyCon experience to those of us who didn't get to go, please >> send me an email. >> >> We hope to plan this quickly, and the Videos are already available. So, >> please email me and let's all get together to make this happen! :) >> >> Cheers, >> >> >> >> Glen >> >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> http://mail.python.org/mailman/listinfo/baypiggies >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From glen at glenjarvis.com Sun Feb 28 05:03:50 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Sat, 27 Feb 2010 20:03:50 -0800 Subject: [Baypiggies] Progress Report: Post PyCon Brain Dump (or whatever we call it) Message-ID: I wanted to share an update on the Post PyCon Brain Dump (or whatever it's going to be called). I envision this to be a place for both openspaces (aka unconferences) and for reviewing our favorite videos. If you will, take a quick moment to see what the open space is about: http://us.pycon.org/2010/openspace/ I also have built a small voting system -- completely anonymous -- on what videos we wish to watch. The URL may change since pycon may be a misleading prefix. But, for now, here is the voting URL: http://pycon.glenjarvis.com/ Yes, with this type of complete anonymous and simple voting, it could be abused. We do still have Web Server logs if we really wanted to troll through and eliminate obvious over-voting. So, click on the talk name to review it, or click "I like" if you like it. It's that simple at this stage. The Log in and Create Account links aren't hooked up yet, but we do have all the backed with data in place and a Django Admin to edit data. It won't take long at all to make a login if people wish to use this to register for the event. We are looking at a Saturday (still undetermined) from 11 am until (still hammering out details). So, obviously, this is not a real announcement, but an effort to show the progress, keep the energy going, and to get volunteers to make this happen. There are about six of us who volunteered to make this happen. Wanna make it one more? I have organized a Google Wave for those who wish to help organize (it's *so* much easier than managing a million emails). My Google Wave account is glen.r.jarvis and I've created a wave with title "Organizing a PostPyCon Meetup." Cheers, Glen Jarvis -------------- next part -------------- An HTML attachment was scrubbed... URL: From glen at glenjarvis.com Sun Feb 28 05:12:40 2010 From: glen at glenjarvis.com (Glen Jarvis) Date: Sat, 27 Feb 2010 20:12:40 -0800 Subject: [Baypiggies] PyCon in SF In-Reply-To: References: Message-ID: OOOPPPsss... Okay, that account has been disabled (since it went to everyone). Alec, are you volunteering? If so, I'll make a different account.. and, err.. I'll try to respond off list :) Sorry for the mistake. Cheers, Glen On Sat, Feb 27, 2010 at 7:41 PM, Glen Jarvis wrote: > Thank you for volunteering to help with organizing a sort of 'after pycon > braindump.' [Actually, this is a form letter and I'm not certain you did > volunteer. You have a great suggestion on picking a tool to pick a date > ahead of time.] There are so many people in this area that are interested, I > think we can get the same type of energy. > > I envision this to be a place for both openspaces (aka unconferences) and > for reviewing our favorite videos. If you will, take a quick moment to see > what the open space is about: > http://us.pycon.org/2010/openspace/ > > I want to get the creative juices flowing, but it also helps to have a > little infrastructure. Since it's only been one day, the emphasis will be on > little. > > I have created a page of all of the videos that I have found. One can also > vote anonymously on the talks you prefer on this page. There's no security, > so someone could vote a million times. But, for this level, I think that's > okay. > > That website is http://pycon.glenjarvis.com/ You will see that: > > * I chose a boring name (pycon and also pycon_dump in places in code). I > need help finding a good name to use in all of these places, > * The links on the page (except for "I like") aren't yet activated > * I hadn't made any account login pages yet > * More details will launch a new window under the subject area > > This was knocked up quickly with Django and I will soon put this in a > repository so others can chip in, edit code, review code, etc. > > Also, the data is in the Admin and I've created an account just for you. > > Go to this URL: > > http://pycon.glenjarvis.com/admin/ > Your username is: alecf > Your password is: )BtjV^5Y > > You are currently super user -- you could wipe everything out -- or add a > million records. Again, for this stage creative juices are more important to > me than data security. There's nothing of consequence (except my hard work > to scrape this data) that is in this database. Passwords are encrypted, but > others organizing will also have super user access. So, don't pick a > password that would compromise you anywhere -- just in case. > > And, finally, it is *MUCH* easier for me to use Google Wave than it is to > manage so many email threads. I know some people have concerns about having > a Google Account. For those people I will manually cut and paste messages to > them. I really hope we can use Google Wave -- it will make my life so much > easier. > > My Google Wave account is glen.r.jarvis and I've created a wave with title > "Organizing a PostPyCon Meetup." > > Feel free to poke around as we start building this from the very very > basics to something awesome? > > Cheers, > > > Glen > > > > On Fri, Feb 26, 2010 at 8:51 AM, Alec Flett wrote: > >> Might I suggest: >> >> http://whenisgood.net/ >> >> for figuring out a day/time.. >> >> Alec >> >> On Fri, Feb 26, 2010 at 7:47 AM, Glen Jarvis wrote: >> >>> We are planning a PyCon in SF get-together where we will watch Pycon >>> Videos, have time for discussion, bring in some food and beverage, and >>> socialize. >>> >>> My experience with planning a group SpamALot is that not everyone wants >>> to be involved in making the decisions for time, location, place, etc. >>> However, some people do. We are building a planning team and we already have >>> three people on the 'committee.' If you wish to be involved in bringing a >>> small slice of the PyCon experience to those of us who didn't get to go, >>> please send me an email. >>> >>> We hope to plan this quickly, and the Videos are already available. So, >>> please email me and let's all get together to make this happen! :) >>> >>> Cheers, >>> >>> >>> >>> Glen >>> >>> _______________________________________________ >>> Baypiggies mailing list >>> Baypiggies at python.org >>> To change your subscription options or unsubscribe: >>> http://mail.python.org/mailman/listinfo/baypiggies >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: