From javier at candeira.com Sun Feb 2 11:15:18 2014 From: javier at candeira.com (Javier Candeira) Date: Sun, 2 Feb 2014 21:15:18 +1100 Subject: [melbourne-pug] February MPUG Meeting: Python-Future, Verification, Timeouts, Metahuristics. Monday 3 February, 6PM, Inspire 9, 41 Stewart St Richmond Message-ID: Dear Melbourne Pythonistas, Tomorrow, Monday 3, we'll hold the February meeting of the Melbourne Python Users Group. Time: 6pm Venue: Inspire 9, 41 Stewart St. Richmond. 50m from Richmond Train Station. Program: * Ed Schofield -- Update on Python-Future for Python 2/3 compatibility * Tennessee Leeuwenburg -- Verification: The Quantitative Science of Knowing how Wrong You Are * Andrew Walker -- How Hard Could it be to Implement Timeouts? * Rory Hart -- Metaheuristics and Python As usual, we'll order pizza, with a $10 contribution required. Drinks are BYO. If you have a tip to share, or want to give a short presentation on a library you've been using lately, please just come forward. The March session is open to all comers! With much appreciation to Inspire 9 for the donation of the use of the venue, The MPUG organisers From dan at acommoncreative.com Mon Feb 3 07:16:30 2014 From: dan at acommoncreative.com (dan) Date: Mon, 3 Feb 2014 17:16:30 +1100 Subject: [melbourne-pug] python GIS volunteers? Message-ID: Hi MPUG, In addition to my day job, I am currently devoting my free time to assisting with the online side of the Public Transport Not Traffic Campaign (http://www.publictransportnottraffic.org/). As it's a community run campaign, funding is reasonably tight and there is essentially no budget to pay for a tech team. So I'm trying to organise a volunteer team of technical professionals who have a desire to contribute their skills and time, in whatever capacity they can, to assist with the execution of the campaign's online strategy. The reason I'm sending this to MPUG is that a key project for the campaign is running on a python based open source GIS platform and thus might be of interest to pythonistas with a GIS bent. If you think you might be interested in getting involved and helping out, or know anyone who would be, please get in touch! Your skills would really make a difference to the success of the campaign. Thanks for reading, Dan Peade PS. Also on the look out for css/html/js and graphic design skills or anyone with a bit of technical nouse, really. -- Common Code = { 'email': 'dan at commoncode.com.au', 'mobile': '0422 987 423', 'address': '114 Hoddle Street, Abbotsford 3067', 'zen': 'http://www.python.org/dev/peps/pep-0020/', } -------------- next part -------------- An HTML attachment was scrubbed... URL: From tleeuwenburg at gmail.com Mon Feb 3 10:47:27 2014 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Mon, 3 Feb 2014 20:47:27 +1100 Subject: [melbourne-pug] Thanks to everyone for their ideas Message-ID: Hi all, Thanks to everyone who contributed their ideas today. I've got an editor window full of keywords to search for and a lot of suggestions for what's good and not so good about what I'm doing. It will definitely help me make the next iteration that I show someone a more impressive and useful tool. Cheers, -Tennessee -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think" -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at python.org Mon Feb 3 17:08:15 2014 From: brian at python.org (Brian Curtin) Date: Mon, 03 Feb 2014 16:08:15 -0000 Subject: [melbourne-pug] PyCon 2014 - 400 tickets remain! Message-ID: Hi Melbourne Python Users Group! PyCon 2014, taking place April 9-17 in Montreal, has just passed 1600 tickets sold, and we're seeing sales ramp up very quickly. Late last week we were announcing the 1500 mark, and with a cap of 2000 tickets, we expect these last four hundred to move quickly. Head to https://us.pycon.org/2014/registration/ to buy yours today! This year's talk schedule is available at https://us.pycon.org/2014/schedule/talks/, along with a keynote series including EFF co-founder John Perry Barlow; IPython creator, Fernando Perez; PSF director, Jessica McKellar; PSF chairman, Van Lindberg; and Python's creator, Guido van Rossum. The tutorial schedule is available at https://us.pycon.org/2014/schedule/tutorials/, and spaces are limited! You'd be hard pressed to find a better value than our tutorials. We've long thought PyCon to be family friendly, and last year's "Young Coder" tutorials were a hugely successful step towards showing it. The tutorials are back again this year, with registration opening shortly for children 12 and over: https://us.pycon.org/2014/events/letslearnpython/. For kids under 12, we're pleased to be offering childcare! See https://us.pycon.org/2014/childcare/ for details. The 5K Charity Fun Run is on its third year, with proceeds benefiting the EFF. It's a fun time and supports a good cause. For the 5K and many other events, check out the Events section on https://us.pycon.org/2014/ If you require an invitation letter in order acquire a travel visa, please email pycon-organizers at python.org! For the latest updates, follow us on Twitter at https://twitter.com/pycon, and on our blog at http://pycon.blogspot.com/ Hope to see you in Montreal! Diana Clarke, Chair diana.joan.clarke at gmail.com Brian Curtin, Publicity Coordinator brian at python.org From brian at microcomaustralia.com.au Mon Feb 3 22:57:10 2014 From: brian at microcomaustralia.com.au (Brian May) Date: Tue, 4 Feb 2014 08:57:10 +1100 Subject: [melbourne-pug] python future debian Message-ID: After last night's talk on Python future, I decided to look up the status of packages for Debian. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=736523 Oops. Seems to have deteriorated into a silly argument over what teamwork means :-( -- Brian May -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Tue Feb 4 00:37:07 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 04 Feb 2014 10:37:07 +1100 Subject: [melbourne-pug] python future debian References: Message-ID: <85a9e7g44c.fsf@benfinney.id.au> Brian May writes: > Oops. Seems to have deteriorated into a silly argument over what > teamwork means :-( Yeah, there are some bull-headed personalities in software projects. Even ones working with Python :-) With transparent projects, like Debian and Python, the discussions are open for everyone to see. The benefit of this is that it helps to curb tendencies to hide bad behaviour. The cost of that is that the remaining bad behaviour is there for all to see. -- \ ?Guaranteed to work throughout its useful life.? ?packaging for | `\ clockwork toy, Hong Kong | _o__) | Ben Finney From javier at candeira.com Tue Feb 4 00:50:27 2014 From: javier at candeira.com (Javier Candeira) Date: Tue, 4 Feb 2014 10:50:27 +1100 Subject: [melbourne-pug] Using Bittorrent for deployments Message-ID: Awesome: http://in.pycon.org/2010/talks/63-deployment-to-large-server-farms-using-fabric-and-bittorrent. J From ben+python at benfinney.id.au Tue Feb 4 01:01:10 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 04 Feb 2014 11:01:10 +1100 Subject: [melbourne-pug] Using Bittorrent for deployments References: Message-ID: <8561ovg309.fsf@benfinney.id.au> Javier Candeira writes: > http://in.pycon.org/2010/talks/63-deployment-to-large-server-farms-using-fabric-and-bittorrent. Sounds great! Is the video or audio recording online for us to download? BitTorrent is a great technology for reversing the usual ?victim of our own success? suffered by a site when everyone suddenly shows up to download something. More non-criminal uses of BitTorrent makes me happy, because they present more ammunition against attempts to unilaterally criminalise and filter BitTorrent traffic. In the same vein, . -- \ ?If you continue running Windows, your system may become | `\ unstable.? ?Microsoft, Windows 95 bluescreen error message | _o__) | Ben Finney From lars at yencken.org Tue Feb 4 03:26:38 2014 From: lars at yencken.org (Lars Yencken) Date: Tue, 4 Feb 2014 13:26:38 +1100 Subject: [melbourne-pug] Using Bittorrent for deployments In-Reply-To: <8561ovg309.fsf@benfinney.id.au> References: <8561ovg309.fsf@benfinney.id.au> Message-ID: Etsy also had a great post on using BitTorrent to replicate Solr search indexes. Worth a read: http://codeascraft.com/2012/01/23/solr-bittorrent-index-replication/ On 4 February 2014 11:01, Ben Finney wrote: > Javier Candeira writes: > > > > http://in.pycon.org/2010/talks/63-deployment-to-large-server-farms-using-fabric-and-bittorrent > . > > Sounds great! Is the video or audio recording online for us to download? > > BitTorrent is a great technology for reversing the usual ?victim of our > own success? suffered by a site when everyone suddenly shows up to > download something. > > More non-criminal uses of BitTorrent makes me happy, because they > present more ammunition against attempts to unilaterally criminalise and > filter BitTorrent traffic. > > In the same vein, . > > -- > \ ?If you continue running Windows, your system may become | > `\ unstable.? ?Microsoft, Windows 95 bluescreen error message | > _o__) | > Ben Finney > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > -------------- next part -------------- An HTML attachment was scrubbed... URL: From javier at candeira.com Tue Feb 4 04:03:00 2014 From: javier at candeira.com (Javier Candeira) Date: Tue, 4 Feb 2014 14:03:00 +1100 Subject: [melbourne-pug] Using Bittorrent for deployments Message-ID: Also via the same thread in the Ansible mailing list (which I can't really recommend if you don't use Ansible, because it's high traffic, but has occasional gems like these), a Murder (of crows): And a Herd On Tue, Feb 4, 2014 at 11:01 AM, Ben Finney wrote: > Javier Candeira writes: > >> http://in.pycon.org/2010/talks/63-deployment-to-large-server-farms-using-fabric-and-bittorrent. > > Sounds great! Is the video or audio recording online for us to download? No idea. I just thought the abstract was informative. J From javier at candeira.com Tue Feb 4 04:05:07 2014 From: javier at candeira.com (Javier Candeira) Date: Tue, 4 Feb 2014 14:05:07 +1100 Subject: [melbourne-pug] Using Bittorrent for deployments In-Reply-To: References: Message-ID: I sent too early. This is what I meant by "herd": "And a Herd of Pythons: https://github.com/russss/Herd" This is one python file. J On Tue, Feb 4, 2014 at 2:03 PM, Javier Candeira wrote: > Also via the same thread in the Ansible mailing list (which I can't > really recommend if you don't use Ansible, because it's high traffic, > but has occasional gems like these), a Murder (of crows): > > > And a Herd > > On Tue, Feb 4, 2014 at 11:01 AM, Ben Finney wrote: >> Javier Candeira writes: >> >>> http://in.pycon.org/2010/talks/63-deployment-to-large-server-farms-using-fabric-and-bittorrent. >> >> Sounds great! Is the video or audio recording online for us to download? > > No idea. I just thought the abstract was informative. > > J From javier at candeira.com Tue Feb 4 04:10:00 2014 From: javier at candeira.com (Javier Candeira) Date: Tue, 4 Feb 2014 14:10:00 +1100 Subject: [melbourne-pug] Using Bittorrent for deployments In-Reply-To: References: Message-ID: Grrr, I can't even use email. And this is what I meant by "a Murder of crows": https://github.com/lg/murder J On Tue, Feb 4, 2014 at 2:05 PM, Javier Candeira wrote: > I sent too early. This is what I meant by "herd": > > "And a Herd of Pythons: https://github.com/russss/Herd" > > This is one python file. > > J > > On Tue, Feb 4, 2014 at 2:03 PM, Javier Candeira wrote: >> Also via the same thread in the Ansible mailing list (which I can't >> really recommend if you don't use Ansible, because it's high traffic, >> but has occasional gems like these), a Murder (of crows): >> >> >> And a Herd >> >> On Tue, Feb 4, 2014 at 11:01 AM, Ben Finney wrote: >>> Javier Candeira writes: >>> >>>> http://in.pycon.org/2010/talks/63-deployment-to-large-server-farms-using-fabric-and-bittorrent. >>> >>> Sounds great! Is the video or audio recording online for us to download? >> >> No idea. I just thought the abstract was informative. >> >> J From javier at candeira.com Tue Feb 4 04:29:16 2014 From: javier at candeira.com (Javier Candeira) Date: Tue, 4 Feb 2014 14:29:16 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? Message-ID: At work I need to manipulate a series of excel spreasheets, and I seem to remember that people have discussed the topic in a meeting. I have two questions to anyone who's already had to do the same: - What's the current best Python library for excel manipulation? - Are there any limitations for handling Excel sheets under Linux? What kind of thing would I need Windows (and presumably, a live copy of Office) for? As background for the question, I'm trying to avoid using Windows at all. My ideal workflow would involve users downloading generated spreadsheets from a webserver. Thanks, Javier From bianca.rachel.gibson at gmail.com Tue Feb 4 04:41:08 2014 From: bianca.rachel.gibson at gmail.com (Bianca Gibson) Date: Tue, 4 Feb 2014 14:41:08 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: What exactly are you using it for? Is csv an option? Bianca -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcrisp at netspace.net.au Tue Feb 4 04:48:15 2014 From: dcrisp at netspace.net.au (David Crisp) Date: Tue, 4 Feb 2014 14:48:15 +1100 (EST) Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: Oops... : This was under Windows, but I beleive the libs work perfeclty under Linux as well.. Regards, David On Tue, 4 Feb 2014, Javier Candeira wrote: > At work I need to manipulate a series of excel spreasheets, and I seem > to remember that people have discussed the topic in a meeting. I have > two questions to anyone who's already had to do the same: > > - What's the current best Python library for excel manipulation? > - Are there any limitations for handling Excel sheets under Linux? > What kind of thing would I need Windows (and presumably, a live copy > of Office) for? > > As background for the question, I'm trying to avoid using Windows at > all. My ideal workflow would involve users downloading generated > spreadsheets from a webserver. > > Thanks, > > Javier > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > From javier at candeira.com Tue Feb 4 04:49:30 2014 From: javier at candeira.com (Javier Candeira) Date: Tue, 4 Feb 2014 14:49:30 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: No, CSV is not an option. I need to generate an Excel that users will run macros on. I can possibly (and hopefully) have a template that already has the macros in it, and just load the new data on the template every time a user hits the "download" button. J On Tue, Feb 4, 2014 at 2:41 PM, Bianca Gibson wrote: > What exactly are you using it for? Is csv an option? > > Bianca > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > From dcrisp at netspace.net.au Tue Feb 4 04:47:44 2014 From: dcrisp at netspace.net.au (David Crisp) Date: Tue, 4 Feb 2014 14:47:44 +1100 (EST) Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: I have just finished a major project using the xlrd and xlwt python libs. Theres also an xlutil lib as well which allows you to manipiulate excel. but i havent played with that one. (I also used pymssql to push this data into an MSSQL database.. but thats a different story) For a sheet that holds basic data I havent found anything the libs wont do. They work in Py3.3 and Py2.7 https://pypi.python.org/pypi?%3Aaction=search&term=xlrd&submit=search Regards, David On Tue, 4 Feb 2014, Javier Candeira wrote: > At work I need to manipulate a series of excel spreasheets, and I seem > to remember that people have discussed the topic in a meeting. I have > two questions to anyone who's already had to do the same: > > - What's the current best Python library for excel manipulation? > - Are there any limitations for handling Excel sheets under Linux? > What kind of thing would I need Windows (and presumably, a live copy > of Office) for? > > As background for the question, I'm trying to avoid using Windows at > all. My ideal workflow would involve users downloading generated > spreadsheets from a webserver. > > Thanks, > > Javier > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > From mattbster at gmail.com Tue Feb 4 04:51:06 2014 From: mattbster at gmail.com (mattb b) Date: Tue, 4 Feb 2014 14:51:06 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: If you can use a csv then life would be easier. It will also depend on which version of excel you are supporting. I have used this in the past and it was good. https://bitbucket.org/ericgazoni/openpyxl/overview On Feb 4, 2014 2:48 PM, "David Crisp" wrote: > Oops... : > > This was under Windows, but I beleive the libs work perfeclty under Linux > as well.. > > Regards, > David > > > On Tue, 4 Feb 2014, Javier Candeira wrote: > > At work I need to manipulate a series of excel spreasheets, and I seem >> to remember that people have discussed the topic in a meeting. I have >> two questions to anyone who's already had to do the same: >> >> - What's the current best Python library for excel manipulation? >> - Are there any limitations for handling Excel sheets under Linux? >> What kind of thing would I need Windows (and presumably, a live copy >> of Office) for? >> >> As background for the question, I'm trying to avoid using Windows at >> all. My ideal workflow would involve users downloading generated >> spreadsheets from a webserver. >> >> Thanks, >> >> Javier >> _______________________________________________ >> melbourne-pug mailing list >> melbourne-pug at python.org >> https://mail.python.org/mailman/listinfo/melbourne-pug >> >> _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hartror at gmail.com Tue Feb 4 05:08:41 2014 From: hartror at gmail.com (Rory Hart) Date: Tue, 4 Feb 2014 15:08:41 +1100 Subject: [melbourne-pug] Travelling Salesman Slides & Code Message-ID: Thanks to all those who watched my talk last night. I've uploaded the code* etc if you would like to take a closer look. Code: https://github.com/hartror/gls_tsp Slides: https://speakerdeck.com/hartror/travelling-salesman-in-python And for extra credit, the paper I based my implementation on: Christos Voudouris, Edward Tsang, Guided local search and its application to the traveling salesman problem, European Journal of Operational Research 113 (1999) 469-499 http://www.ceet.niu.edu/faculty/ghrayeb/IENG576s04/papers/Local%20Search/local%20search%20for%20tsp.pdf * Note the code was written for ease of demoing and explanation rather than performance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From miked at dewhirst.com.au Tue Feb 4 05:15:13 2014 From: miked at dewhirst.com.au (Mike Dewhirst) Date: Tue, 04 Feb 2014 15:15:13 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: <52F06951.4070102@dewhirst.com.au> On 4/02/2014 2:29pm, Javier Candeira wrote: > At work I need to manipulate a series of excel spreasheets, and I seem > to remember that people have discussed the topic in a meeting. I have > two questions to anyone who's already had to do the same: > > - What's the current best Python library for excel manipulation? > - Are there any limitations for handling Excel sheets under Linux? > What kind of thing would I need Windows (and presumably, a live copy > of Office) for? I haven't used MS Office for years but have no difficulty using LibreOffice (and OpenOffice before that) to produce Excel spreadsheets. The spreadsheets are just data so should be viable on Windows even if produced on Linux. All you need is a tame Windows user to prove it works. My wife still uses MS Office from last century so I could prove an old .xls format spreadsheet for you. I could also prove the latest .xlsx format works on Windows via LibreOffice but that isn't MS Office. I know this isn't a python library as you mentioned but LibreOffice is a mechanism for producing Excel spreadsheets for testing or getting you some fast results while perfecting the python library side of things. Also I believe LibreOffice has a Python interface - probably for the macros - although I have never bothered trying it out. Mike > > As background for the question, I'm trying to avoid using Windows at > all. My ideal workflow would involve users downloading generated > spreadsheets from a webserver. > > Thanks, > > Javier > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > From deeprave at gmail.com Tue Feb 4 05:19:10 2014 From: deeprave at gmail.com (David Nugent) Date: Tue, 04 Feb 2014 15:19:10 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: They work very well. I Have been using xlrd for some years under both python 2. 7 and 3.3 on OSX and Linux over some years now whenever I have had to deal with them. The down side is that they do not understand the contemporary version of excel but need the saved in the previous document format, but this is at least only a minor inconvenience On 4 February 2014 14:48:15 GMT+11:00, David Crisp wrote: >Oops... : > >This was under Windows, but I beleive the libs work perfeclty under >Linux >as well.. > >Regards, >David > > >On Tue, 4 Feb 2014, Javier Candeira wrote: > >> At work I need to manipulate a series of excel spreasheets, and I >seem >> to remember that people have discussed the topic in a meeting. I have >> two questions to anyone who's already had to do the same: >> >> - What's the current best Python library for excel manipulation? >> - Are there any limitations for handling Excel sheets under Linux? >> What kind of thing would I need Windows (and presumably, a live copy >> of Office) for? >> >> As background for the question, I'm trying to avoid using Windows at >> all. My ideal workflow would involve users downloading generated >> spreadsheets from a webserver. >> >> Thanks, >> >> Javier >> _______________________________________________ >> melbourne-pug mailing list >> melbourne-pug at python.org >> https://mail.python.org/mailman/listinfo/melbourne-pug >> >_______________________________________________ >melbourne-pug mailing list >melbourne-pug at python.org >https://mail.python.org/mailman/listinfo/melbourne-pug -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim at growthpath.com.au Tue Feb 4 05:19:39 2014 From: tim at growthpath.com.au (Tim Richardson) Date: Tue, 4 Feb 2014 15:19:39 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: On Tue, Feb 4, 2014 at 2:47 PM, David Crisp wrote: > I have just finished a major project using the xlrd and xlwt python libs. > Theres also an xlutil lib as well which allows you to manipiulate excel. > but i havent played with that one. (I also used pymssql to push this data > into an MSSQL database.. but thats a different story) > > For a sheet that holds basic data I havent found anything the libs wont do. > > They work in Py3.3 and Py2.7 > +1 from me. Just in case it wasn't clear, these are pure python modules; they don't interact with Office automation objects, so they run anywhere (linux), it's more robust than using Office and much faster. However, these libraries offers only a limited subset of Excel functionality; you can't make charts, pivot tables or macros. The idea of opening an existing document, and modifying it, is not really supported as far as I could work out. You create new sheets. I may be wrong about that; all I've used it for is creating spreadsheets. I suspect google docs has a much better API in terms of enabling maximum "richness" and it may be something worth thinking about (i.e. Google Spreadsheets). The only real way to get perfect Excel creation is to automate an office binary installation. -- *Tim Richardson, Director* GrowthPath, Data-driven profit growth Mobile: +61 423 091 732 Office: +61 3 8678 1850 I tweet useful business & IT tips at growthpath_au GrowthPath Pty Ltd ABN 76 133 733 963 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rasjidw at openminddev.net Tue Feb 4 05:22:20 2014 From: rasjidw at openminddev.net (Rasjid Wilcox) Date: Tue, 04 Feb 2014 15:22:20 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: <52F06AFC.2080201@openminddev.net> Hi Javier, I use xlwt (https://pypi.python.org/pypi/xlwt) to generate XLS spreadsheets. It is a fork of pyExcelerator which is not longer maintained. For fairly standard spreadsheets it works well. You can do basic formatting, both in terms of fonts (bold etc), and also in terms of numeric formatting (currency and date formats etc). Last I checked (and I don't think this has changed), it does not do charts or fancy stuff like that. As long as it meets your needs it terms of spreadsheet complexity, it will work well. Runs on any platform (will be fine on a Linux web-server), and is fast. I looked into this extensively a few years ago - at the time it was definitely the way to go as long as it meets your requirements in terms of capabilities. Your other options are: Under Linux etc. Use a head-less copy of OpenOffice, and use its automation (pyUno) to generate a spreadsheet that way. Be prepared for a lot of pain. Use Jython (or bridge to java in some way) and use one of the Java libraries to generate XLS files. At least one of these has basic support for charts (although you may need a base template file to work from). I know there are some .Net libraries that can also generate xlsx files - these may work under Mono and IronPython. Under Windows: All of the above, with the bonus that any .Net libraries are definitely going to work. Automate MS Office using Python's COM support. The pro's of using automation (either OpenOffice or MS Office) is that you will get access to a much greater range of features. It will, however, be slow, and can take noticeable amounts of time to generate a large spreadsheet. Any of the libraries that generate the file directly will be *much* faster, and a better fit for server-side deployment. Hmm... just seen your last email regarding macros. Not sure if xlwt supports that. You may have luck with the xlutils library mentioned - otherwise you may need to move to one of the more heavy duty options - either using a Java or .Net / Mono library, or going for full automation (I'd suggest MS Office over OpenOffice if going that route). Cheers, Rasjid. On 4/02/2014 2:29 PM, Javier Candeira wrote: > At work I need to manipulate a series of excel spreasheets, and I seem > to remember that people have discussed the topic in a meeting. I have > two questions to anyone who's already had to do the same: > > - What's the current best Python library for excel manipulation? > - Are there any limitations for handling Excel sheets under Linux? > What kind of thing would I need Windows (and presumably, a live copy > of Office) for? > > As background for the question, I'm trying to avoid using Windows at > all. My ideal workflow would involve users downloading generated > spreadsheets from a webserver. > > Thanks, > > Javier > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug From javier at candeira.com Tue Feb 4 05:38:04 2014 From: javier at candeira.com (Javier Candeira) Date: Tue, 4 Feb 2014 15:38:04 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: <52F06AFC.2080201@openminddev.net> References: <52F06AFC.2080201@openminddev.net> Message-ID: Thanks everyone. Some more background and answers: - I'm a LibreOffice fan (this is me after a love night with two LibreOffice hackers [1]), but this has to be MS Office. I'm automating an entrenched reporting process that produces pixel-perfect A4 documents for printing, and I can't recreate the accumulated work expressed in the existing sheets. - I'm also a Google Spreadsheets fan, to the point where I have contributed to a library for accessing it. [2] But same as above. - xlwt or xlutil might work if it allows me to manipulate existing spreadsheets. If not, I'd try creating my reporting template and linking it to sheets with sample data, and to recreate the data-bearing sheets every week/month/whenever. - I don't really need to generate macros or charts, because there are templates that have them already. I really only have to put in data in an existing database (and munge the incoming data a bit before writing the neat rows and columns). [1] http://www.flickr.com/photos/hiperactivo/4364689348/ [2] https://github.com/burnash/gspread Thanks again, and sorry for not giving enough background at the start of the thread. J On Tue, Feb 4, 2014 at 3:22 PM, Rasjid Wilcox wrote: > Hi Javier, > > I use xlwt (https://pypi.python.org/pypi/xlwt) to generate XLS spreadsheets. > It is a fork of pyExcelerator which is not longer maintained. > > For fairly standard spreadsheets it works well. You can do basic > formatting, both in terms of fonts (bold etc), and also in terms of numeric > formatting (currency and date formats etc). > > Last I checked (and I don't think this has changed), it does not do charts > or fancy stuff like that. > > As long as it meets your needs it terms of spreadsheet complexity, it will > work well. Runs on any platform (will be fine on a Linux web-server), and > is fast. > > I looked into this extensively a few years ago - at the time it was > definitely the way to go as long as it meets your requirements in terms of > capabilities. > > > Your other options are: > > Under Linux etc. > Use a head-less copy of OpenOffice, and use its automation (pyUno) to > generate a spreadsheet that way. Be prepared for a lot of pain. > Use Jython (or bridge to java in some way) and use one of the Java > libraries to generate XLS files. At least one of these has basic support > for charts (although you may need a base template file to work from). > > I know there are some .Net libraries that can also generate xlsx files - > these may work under Mono and IronPython. > > > Under Windows: > All of the above, with the bonus that any .Net libraries are definitely > going to work. > Automate MS Office using Python's COM support. > > > The pro's of using automation (either OpenOffice or MS Office) is that you > will get access to a much greater range of features. It will, however, be > slow, and can take noticeable amounts of time to generate a large > spreadsheet. > > Any of the libraries that generate the file directly will be *much* faster, > and a better fit for server-side deployment. > > > > Hmm... just seen your last email regarding macros. Not sure if xlwt > supports that. You may have luck with the xlutils library mentioned - > otherwise you may need to move to one of the more heavy duty options - > either using a Java or .Net / Mono library, or going for full automation > (I'd suggest MS Office over OpenOffice if going that route). > > > Cheers, > > Rasjid. > > > > > > > > On 4/02/2014 2:29 PM, Javier Candeira wrote: >> >> At work I need to manipulate a series of excel spreasheets, and I seem >> to remember that people have discussed the topic in a meeting. I have >> two questions to anyone who's already had to do the same: >> >> - What's the current best Python library for excel manipulation? >> - Are there any limitations for handling Excel sheets under Linux? >> What kind of thing would I need Windows (and presumably, a live copy >> of Office) for? >> >> As background for the question, I'm trying to avoid using Windows at >> all. My ideal workflow would involve users downloading generated >> spreadsheets from a webserver. >> >> Thanks, >> >> Javier >> _______________________________________________ >> >> melbourne-pug mailing list >> melbourne-pug at python.org >> https://mail.python.org/mailman/listinfo/melbourne-pug > > > _______________________________________________ > > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug From javier at candeira.com Tue Feb 4 05:40:17 2014 From: javier at candeira.com (Javier Candeira) Date: Tue, 4 Feb 2014 15:40:17 +1100 Subject: [melbourne-pug] Travelling Salesman Slides & Code In-Reply-To: References: Message-ID: Hi Rory, Thanks a lot for the presentation and for the extra info! J From tim at growthpath.com.au Tue Feb 4 05:42:12 2014 From: tim at growthpath.com.au (Tim Richardson) Date: Tue, 4 Feb 2014 15:42:12 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: <52F06AFC.2080201@openminddev.net> Message-ID: There is another way, which I have also used. You can issue a spreadsheet with a VBA-embedded query which fetches data from a web-service (VBA support for this is very stone-age but you can make it work), or use a view on a database server and query that (also via an embedded macro). That is, you make the spreadsheet the client, basically. On Tue, Feb 4, 2014 at 3:38 PM, Javier Candeira wrote: > Thanks everyone. Some more background and answers: > > - I'm a LibreOffice fan (this is me after a love night with two > LibreOffice hackers [1]), but this has to be MS Office. I'm automating > an entrenched reporting process that produces pixel-perfect A4 > documents for printing, and I can't recreate the accumulated work > expressed in the existing sheets. > - I'm also a Google Spreadsheets fan, to the point where I have > contributed to a library for accessing it. [2] But same as above. > - xlwt or xlutil might work if it allows me to manipulate existing > spreadsheets. If not, I'd try creating my reporting template and > linking it to sheets with sample data, and to recreate the > data-bearing sheets every week/month/whenever. > - I don't really need to generate macros or charts, because there are > templates that have them already. I really only have to put in data in > an existing database (and munge the incoming data a bit before writing > the neat rows and columns). > > [1] http://www.flickr.com/photos/hiperactivo/4364689348/ > [2] https://github.com/burnash/gspread > > Thanks again, and sorry for not giving enough background at the start > of the thread. > > J > > On Tue, Feb 4, 2014 at 3:22 PM, Rasjid Wilcox > wrote: > > Hi Javier, > > > > I use xlwt (https://pypi.python.org/pypi/xlwt) to generate XLS > spreadsheets. > > It is a fork of pyExcelerator which is not longer maintained. > > > > For fairly standard spreadsheets it works well. You can do basic > > formatting, both in terms of fonts (bold etc), and also in terms of > numeric > > formatting (currency and date formats etc). > > > > Last I checked (and I don't think this has changed), it does not do > charts > > or fancy stuff like that. > > > > As long as it meets your needs it terms of spreadsheet complexity, it > will > > work well. Runs on any platform (will be fine on a Linux web-server), > and > > is fast. > > > > I looked into this extensively a few years ago - at the time it was > > definitely the way to go as long as it meets your requirements in terms > of > > capabilities. > > > > > > Your other options are: > > > > Under Linux etc. > > Use a head-less copy of OpenOffice, and use its automation (pyUno) to > > generate a spreadsheet that way. Be prepared for a lot of pain. > > Use Jython (or bridge to java in some way) and use one of the Java > > libraries to generate XLS files. At least one of these has basic support > > for charts (although you may need a base template file to work from). > > > > I know there are some .Net libraries that can also generate xlsx files > - > > these may work under Mono and IronPython. > > > > > > Under Windows: > > All of the above, with the bonus that any .Net libraries are definitely > > going to work. > > Automate MS Office using Python's COM support. > > > > > > The pro's of using automation (either OpenOffice or MS Office) is that > you > > will get access to a much greater range of features. It will, however, > be > > slow, and can take noticeable amounts of time to generate a large > > spreadsheet. > > > > Any of the libraries that generate the file directly will be *much* > faster, > > and a better fit for server-side deployment. > > > > > > > > Hmm... just seen your last email regarding macros. Not sure if xlwt > > supports that. You may have luck with the xlutils library mentioned - > > otherwise you may need to move to one of the more heavy duty options - > > either using a Java or .Net / Mono library, or going for full automation > > (I'd suggest MS Office over OpenOffice if going that route). > > > > > > Cheers, > > > > Rasjid. > > > > > > > > > > > > > > > > On 4/02/2014 2:29 PM, Javier Candeira wrote: > >> > >> At work I need to manipulate a series of excel spreasheets, and I seem > >> to remember that people have discussed the topic in a meeting. I have > >> two questions to anyone who's already had to do the same: > >> > >> - What's the current best Python library for excel manipulation? > >> - Are there any limitations for handling Excel sheets under Linux? > >> What kind of thing would I need Windows (and presumably, a live copy > >> of Office) for? > >> > >> As background for the question, I'm trying to avoid using Windows at > >> all. My ideal workflow would involve users downloading generated > >> spreadsheets from a webserver. > >> > >> Thanks, > >> > >> Javier > >> _______________________________________________ > >> > >> melbourne-pug mailing list > >> melbourne-pug at python.org > >> https://mail.python.org/mailman/listinfo/melbourne-pug > > > > > > _______________________________________________ > > > > melbourne-pug mailing list > > melbourne-pug at python.org > > https://mail.python.org/mailman/listinfo/melbourne-pug > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > -- *Tim Richardson, Director* GrowthPath, Data-driven profit growth Mobile: +61 423 091 732 Office: +61 3 8678 1850 I tweet useful business & IT tips at growthpath_au GrowthPath Pty Ltd ABN 76 133 733 963 -------------- next part -------------- An HTML attachment was scrubbed... URL: From henry.walshaw at gmail.com Tue Feb 4 05:48:47 2014 From: henry.walshaw at gmail.com (Henry Walshaw) Date: Tue, 4 Feb 2014 15:48:47 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: <52F06AFC.2080201@openminddev.net> Message-ID: Hi Javier, Like Matt below I'd recommend taking a look at openpyxl (http://pythonhosted.org/openpyxl/) We've used it a lot recently and it does give a great deal over formatting of the outgoing Excel workbook. Unfortunately it does not support reading images and charts from an existing workbook, so they would be lost if you tried reading an existing workbook as a template (openpyxl works with Excel data in memory and only accesses the file handler on save or load, which means that modifying an existing workbook is a case of reading the data in, updating the data and then saving a new excel file over the original one). However I think it's a good choice if all you're doing is manipulating workbooks full of data that the template is applied to later. Cheers -Henry On 4 Feb 2014, at 3:38 pm, Javier Candeira wrote: > Thanks everyone. Some more background and answers: > > - I'm a LibreOffice fan (this is me after a love night with two > LibreOffice hackers [1]), but this has to be MS Office. I'm automating > an entrenched reporting process that produces pixel-perfect A4 > documents for printing, and I can't recreate the accumulated work > expressed in the existing sheets. > - I'm also a Google Spreadsheets fan, to the point where I have > contributed to a library for accessing it. [2] But same as above. > - xlwt or xlutil might work if it allows me to manipulate existing > spreadsheets. If not, I'd try creating my reporting template and > linking it to sheets with sample data, and to recreate the > data-bearing sheets every week/month/whenever. > - I don't really need to generate macros or charts, because there are > templates that have them already. I really only have to put in data in > an existing database (and munge the incoming data a bit before writing > the neat rows and columns). > > [1] http://www.flickr.com/photos/hiperactivo/4364689348/ > [2] https://github.com/burnash/gspread > > Thanks again, and sorry for not giving enough background at the start > of the thread. > > J > > On Tue, Feb 4, 2014 at 3:22 PM, Rasjid Wilcox wrote: >> Hi Javier, >> >> I use xlwt (https://pypi.python.org/pypi/xlwt) to generate XLS spreadsheets. >> It is a fork of pyExcelerator which is not longer maintained. >> >> For fairly standard spreadsheets it works well. You can do basic >> formatting, both in terms of fonts (bold etc), and also in terms of numeric >> formatting (currency and date formats etc). >> >> Last I checked (and I don't think this has changed), it does not do charts >> or fancy stuff like that. >> >> As long as it meets your needs it terms of spreadsheet complexity, it will >> work well. Runs on any platform (will be fine on a Linux web-server), and >> is fast. >> >> I looked into this extensively a few years ago - at the time it was >> definitely the way to go as long as it meets your requirements in terms of >> capabilities. >> >> >> Your other options are: >> >> Under Linux etc. >> Use a head-less copy of OpenOffice, and use its automation (pyUno) to >> generate a spreadsheet that way. Be prepared for a lot of pain. >> Use Jython (or bridge to java in some way) and use one of the Java >> libraries to generate XLS files. At least one of these has basic support >> for charts (although you may need a base template file to work from). >> >> I know there are some .Net libraries that can also generate xlsx files - >> these may work under Mono and IronPython. >> >> >> Under Windows: >> All of the above, with the bonus that any .Net libraries are definitely >> going to work. >> Automate MS Office using Python's COM support. >> >> >> The pro's of using automation (either OpenOffice or MS Office) is that you >> will get access to a much greater range of features. It will, however, be >> slow, and can take noticeable amounts of time to generate a large >> spreadsheet. >> >> Any of the libraries that generate the file directly will be *much* faster, >> and a better fit for server-side deployment. >> >> >> >> Hmm... just seen your last email regarding macros. Not sure if xlwt >> supports that. You may have luck with the xlutils library mentioned - >> otherwise you may need to move to one of the more heavy duty options - >> either using a Java or .Net / Mono library, or going for full automation >> (I'd suggest MS Office over OpenOffice if going that route). >> >> >> Cheers, >> >> Rasjid. >> >> >> >> >> >> >> >> On 4/02/2014 2:29 PM, Javier Candeira wrote: >>> >>> At work I need to manipulate a series of excel spreasheets, and I seem >>> to remember that people have discussed the topic in a meeting. I have >>> two questions to anyone who's already had to do the same: >>> >>> - What's the current best Python library for excel manipulation? >>> - Are there any limitations for handling Excel sheets under Linux? >>> What kind of thing would I need Windows (and presumably, a live copy >>> of Office) for? >>> >>> As background for the question, I'm trying to avoid using Windows at >>> all. My ideal workflow would involve users downloading generated >>> spreadsheets from a webserver. >>> >>> Thanks, >>> >>> Javier >>> _______________________________________________ >>> >>> melbourne-pug mailing list >>> melbourne-pug at python.org >>> https://mail.python.org/mailman/listinfo/melbourne-pug >> >> >> _______________________________________________ >> >> melbourne-pug mailing list >> melbourne-pug at python.org >> https://mail.python.org/mailman/listinfo/melbourne-pug > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From javier at candeira.com Tue Feb 4 05:53:41 2014 From: javier at candeira.com (Javier Candeira) Date: Tue, 4 Feb 2014 15:53:41 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: <52F06AFC.2080201@openminddev.net> Message-ID: Then, just to be clear: there doesn't seem to exist a Python library that will roundtrip an excel sheet to its original state just by reading it in and writing it again, without modification. Correct? J On Tue, Feb 4, 2014 at 3:48 PM, Henry Walshaw wrote: > Hi Javier, > > Like Matt below I'd recommend taking a look at openpyxl (http://pythonhosted.org/openpyxl/) We've used it a lot recently and it does give a great deal over formatting of the outgoing Excel workbook. Unfortunately it does not support reading images and charts from an existing workbook, so they would be lost if you tried reading an existing workbook as a template (openpyxl works with Excel data in memory and only accesses the file handler on save or load, which means that modifying an existing workbook is a case of reading the data in, updating the data and then saving a new excel file over the original one). However I think it's a good choice if all you're doing is manipulating workbooks full of data that the template is applied to later. > > Cheers > -Henry > > > On 4 Feb 2014, at 3:38 pm, Javier Candeira wrote: > >> Thanks everyone. Some more background and answers: >> >> - I'm a LibreOffice fan (this is me after a love night with two >> LibreOffice hackers [1]), but this has to be MS Office. I'm automating >> an entrenched reporting process that produces pixel-perfect A4 >> documents for printing, and I can't recreate the accumulated work >> expressed in the existing sheets. >> - I'm also a Google Spreadsheets fan, to the point where I have >> contributed to a library for accessing it. [2] But same as above. >> - xlwt or xlutil might work if it allows me to manipulate existing >> spreadsheets. If not, I'd try creating my reporting template and >> linking it to sheets with sample data, and to recreate the >> data-bearing sheets every week/month/whenever. >> - I don't really need to generate macros or charts, because there are >> templates that have them already. I really only have to put in data in >> an existing database (and munge the incoming data a bit before writing >> the neat rows and columns). >> >> [1] http://www.flickr.com/photos/hiperactivo/4364689348/ >> [2] https://github.com/burnash/gspread >> >> Thanks again, and sorry for not giving enough background at the start >> of the thread. >> >> J >> >> On Tue, Feb 4, 2014 at 3:22 PM, Rasjid Wilcox wrote: >>> Hi Javier, >>> >>> I use xlwt (https://pypi.python.org/pypi/xlwt) to generate XLS spreadsheets. >>> It is a fork of pyExcelerator which is not longer maintained. >>> >>> For fairly standard spreadsheets it works well. You can do basic >>> formatting, both in terms of fonts (bold etc), and also in terms of numeric >>> formatting (currency and date formats etc). >>> >>> Last I checked (and I don't think this has changed), it does not do charts >>> or fancy stuff like that. >>> >>> As long as it meets your needs it terms of spreadsheet complexity, it will >>> work well. Runs on any platform (will be fine on a Linux web-server), and >>> is fast. >>> >>> I looked into this extensively a few years ago - at the time it was >>> definitely the way to go as long as it meets your requirements in terms of >>> capabilities. >>> >>> >>> Your other options are: >>> >>> Under Linux etc. >>> Use a head-less copy of OpenOffice, and use its automation (pyUno) to >>> generate a spreadsheet that way. Be prepared for a lot of pain. >>> Use Jython (or bridge to java in some way) and use one of the Java >>> libraries to generate XLS files. At least one of these has basic support >>> for charts (although you may need a base template file to work from). >>> >>> I know there are some .Net libraries that can also generate xlsx files - >>> these may work under Mono and IronPython. >>> >>> >>> Under Windows: >>> All of the above, with the bonus that any .Net libraries are definitely >>> going to work. >>> Automate MS Office using Python's COM support. >>> >>> >>> The pro's of using automation (either OpenOffice or MS Office) is that you >>> will get access to a much greater range of features. It will, however, be >>> slow, and can take noticeable amounts of time to generate a large >>> spreadsheet. >>> >>> Any of the libraries that generate the file directly will be *much* faster, >>> and a better fit for server-side deployment. >>> >>> >>> >>> Hmm... just seen your last email regarding macros. Not sure if xlwt >>> supports that. You may have luck with the xlutils library mentioned - >>> otherwise you may need to move to one of the more heavy duty options - >>> either using a Java or .Net / Mono library, or going for full automation >>> (I'd suggest MS Office over OpenOffice if going that route). >>> >>> >>> Cheers, >>> >>> Rasjid. >>> >>> >>> >>> >>> >>> >>> >>> On 4/02/2014 2:29 PM, Javier Candeira wrote: >>>> >>>> At work I need to manipulate a series of excel spreasheets, and I seem >>>> to remember that people have discussed the topic in a meeting. I have >>>> two questions to anyone who's already had to do the same: >>>> >>>> - What's the current best Python library for excel manipulation? >>>> - Are there any limitations for handling Excel sheets under Linux? >>>> What kind of thing would I need Windows (and presumably, a live copy >>>> of Office) for? >>>> >>>> As background for the question, I'm trying to avoid using Windows at >>>> all. My ideal workflow would involve users downloading generated >>>> spreadsheets from a webserver. >>>> >>>> Thanks, >>>> >>>> Javier >>>> _______________________________________________ >>>> >>>> melbourne-pug mailing list >>>> melbourne-pug at python.org >>>> https://mail.python.org/mailman/listinfo/melbourne-pug >>> >>> >>> _______________________________________________ >>> >>> melbourne-pug mailing list >>> melbourne-pug at python.org >>> https://mail.python.org/mailman/listinfo/melbourne-pug >> _______________________________________________ >> melbourne-pug mailing list >> melbourne-pug at python.org >> https://mail.python.org/mailman/listinfo/melbourne-pug > > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > From simoncropper at fossworkflowguides.com Tue Feb 4 05:20:44 2014 From: simoncropper at fossworkflowguides.com (Simon Cropper) Date: Tue, 04 Feb 2014 15:20:44 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: <52F06A9C.80002@fossworkflowguides.com> On 04/02/14 14:49, Javier Candeira wrote: > No, CSV is not an option. I need to generate an Excel that users will > run macros on. I can possibly (and hopefully) have a template that > already has the macros in it, and just load the new data on the > template every time a user hits the "download" button. > > J > > On Tue, Feb 4, 2014 at 2:41 PM, Bianca Gibson > wrote: >> What exactly are you using it for? Is csv an option? >> >> Bianca >> >> _______________________________________________ >> melbourne-pug mailing list >> melbourne-pug at python.org >> https://mail.python.org/mailman/listinfo/melbourne-pug >> > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > J, This is a VBA problem not a python problem Your template needs to eat something and whether it is an excel file or a CSV is irrelevant. At some point a convert or import command needs to get your data into your spreadsheet. The only way python can do this is (i) through automation (e.g. http://msdn.microsoft.com/en-us/library/ms178745%28v=vs.90%29.aspx), or (2) by creating a Excel File of data with macros already inserted. Otherwise python is only used to create the datafile and whether this is CSV, dBase or Excel format is irrelevant as long as the process of importing into the template can be automated. -- Cheers Simon Simon Cropper - Open Content Creator Free and Open Source Software Workflow Guides ------------------------------------------------------------ Introduction http://www.fossworkflowguides.com GIS Packages http://www.fossworkflowguides.com/gis bash / Python http://www.fossworkflowguides.com/scripting From svalqui at unimelb.edu.au Tue Feb 4 05:02:54 2014 From: svalqui at unimelb.edu.au (Sergio Valqui) Date: Tue, 4 Feb 2014 04:02:54 +0000 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: Message-ID: <55C5A24A82E95D428C3C6E006DDFFFC13FD43411@000S-EX-MBX-NP1.unimelb.edu.au> Hi All, I use the below, just for basic reading of xls works well. http://code.activestate.com/pypm/xlrd3/ https://github.com/JoiDegn/xlrd3 1. Not sure if is the best, but reads xls 2. Limitations: does not work with xlsx, not sure about macros. 3. You do not need windows but the users would need an application to open xls (MS Office ?) 4. Looks like: from xlrd3 import open_workbook FO_XLSSpreadSheet = open_workbook("PatchSchedule.xls") print ('Opening workbook PatchSchedule.xls...') FO_XLSSheettoReview = FO_XLSSpreadSheet.sheet_by_name('Sheet1') # print values of the first column (0) for all rows for fn_RowIndex in range(FO_XLSSheettoReview.nrows): # if all the cell have a text or number value if FO_XLSSheettoReview.cell(fn_RowIndex,0).ctype == 1: print (FO_XLSSheettoReview.cell(fn_RowIndex,0).value) elif FO_XLSSheettoReview.cell(fn_RowIndex,0).ctype == 2: print(str(FO_XLSSheettoReview.cell(fn_RowIndex,0).value)) else: print ('? Find my type') All the best Sergio. -----Original Message----- From: melbourne-pug [mailto:melbourne-pug-bounces+svalqui=unimelb.edu.au at python.org] On Behalf Of Javier Candeira Sent: Tuesday, 4 February 2014 2:29 PM To: Melbourne Python Users Group Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? At work I need to manipulate a series of excel spreasheets, and I seem to remember that people have discussed the topic in a meeting. I have two questions to anyone who's already had to do the same: - What's the current best Python library for excel manipulation? - Are there any limitations for handling Excel sheets under Linux? What kind of thing would I need Windows (and presumably, a live copy of Office) for? As background for the question, I'm trying to avoid using Windows at all. My ideal workflow would involve users downloading generated spreadsheets from a webserver. Thanks, Javier _______________________________________________ melbourne-pug mailing list melbourne-pug at python.org https://mail.python.org/mailman/listinfo/melbourne-pug From henry.walshaw at gmail.com Tue Feb 4 06:13:42 2014 From: henry.walshaw at gmail.com (Henry Walshaw) Date: Tue, 4 Feb 2014 16:13:42 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: <52F06AFC.2080201@openminddev.net> Message-ID: Javier, The only thing I could think of trying would be pyodbc (assuing you're on a Windows machine, which I know you were trying to avoid) and trying to write through an ODBC connection. Something like: cnxn=pyodbc.connect("""Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)}; Dbq=c:\\datafiles\\XLSfile.xls; readonly=0""", autocommit=True) I haven't tested this though, so it's really just clutching at straws. Otherwise I'm not sure of any Python files that modify Excel in place. Good luck! Cheers, Henry On 4 Feb 2014, at 3:53 pm, Javier Candeira wrote: > Then, just to be clear: there doesn't seem to exist a Python library > that will roundtrip an excel sheet to its original state just by > reading it in and writing it again, without modification. Correct? > > J > > On Tue, Feb 4, 2014 at 3:48 PM, Henry Walshaw wrote: >> Hi Javier, >> >> Like Matt below I'd recommend taking a look at openpyxl (http://pythonhosted.org/openpyxl/) We've used it a lot recently and it does give a great deal over formatting of the outgoing Excel workbook. Unfortunately it does not support reading images and charts from an existing workbook, so they would be lost if you tried reading an existing workbook as a template (openpyxl works with Excel data in memory and only accesses the file handler on save or load, which means that modifying an existing workbook is a case of reading the data in, updating the data and then saving a new excel file over the original one). However I think it's a good choice if all you're doing is manipulating workbooks full of data that the template is applied to later. >> >> Cheers >> -Henry >> >> >> On 4 Feb 2014, at 3:38 pm, Javier Candeira wrote: >> >>> Thanks everyone. Some more background and answers: >>> >>> - I'm a LibreOffice fan (this is me after a love night with two >>> LibreOffice hackers [1]), but this has to be MS Office. I'm automating >>> an entrenched reporting process that produces pixel-perfect A4 >>> documents for printing, and I can't recreate the accumulated work >>> expressed in the existing sheets. >>> - I'm also a Google Spreadsheets fan, to the point where I have >>> contributed to a library for accessing it. [2] But same as above. >>> - xlwt or xlutil might work if it allows me to manipulate existing >>> spreadsheets. If not, I'd try creating my reporting template and >>> linking it to sheets with sample data, and to recreate the >>> data-bearing sheets every week/month/whenever. >>> - I don't really need to generate macros or charts, because there are >>> templates that have them already. I really only have to put in data in >>> an existing database (and munge the incoming data a bit before writing >>> the neat rows and columns). >>> >>> [1] http://www.flickr.com/photos/hiperactivo/4364689348/ >>> [2] https://github.com/burnash/gspread >>> >>> Thanks again, and sorry for not giving enough background at the start >>> of the thread. >>> >>> J >>> >>> On Tue, Feb 4, 2014 at 3:22 PM, Rasjid Wilcox wrote: >>>> Hi Javier, >>>> >>>> I use xlwt (https://pypi.python.org/pypi/xlwt) to generate XLS spreadsheets. >>>> It is a fork of pyExcelerator which is not longer maintained. >>>> >>>> For fairly standard spreadsheets it works well. You can do basic >>>> formatting, both in terms of fonts (bold etc), and also in terms of numeric >>>> formatting (currency and date formats etc). >>>> >>>> Last I checked (and I don't think this has changed), it does not do charts >>>> or fancy stuff like that. >>>> >>>> As long as it meets your needs it terms of spreadsheet complexity, it will >>>> work well. Runs on any platform (will be fine on a Linux web-server), and >>>> is fast. >>>> >>>> I looked into this extensively a few years ago - at the time it was >>>> definitely the way to go as long as it meets your requirements in terms of >>>> capabilities. >>>> >>>> >>>> Your other options are: >>>> >>>> Under Linux etc. >>>> Use a head-less copy of OpenOffice, and use its automation (pyUno) to >>>> generate a spreadsheet that way. Be prepared for a lot of pain. >>>> Use Jython (or bridge to java in some way) and use one of the Java >>>> libraries to generate XLS files. At least one of these has basic support >>>> for charts (although you may need a base template file to work from). >>>> >>>> I know there are some .Net libraries that can also generate xlsx files - >>>> these may work under Mono and IronPython. >>>> >>>> >>>> Under Windows: >>>> All of the above, with the bonus that any .Net libraries are definitely >>>> going to work. >>>> Automate MS Office using Python's COM support. >>>> >>>> >>>> The pro's of using automation (either OpenOffice or MS Office) is that you >>>> will get access to a much greater range of features. It will, however, be >>>> slow, and can take noticeable amounts of time to generate a large >>>> spreadsheet. >>>> >>>> Any of the libraries that generate the file directly will be *much* faster, >>>> and a better fit for server-side deployment. >>>> >>>> >>>> >>>> Hmm... just seen your last email regarding macros. Not sure if xlwt >>>> supports that. You may have luck with the xlutils library mentioned - >>>> otherwise you may need to move to one of the more heavy duty options - >>>> either using a Java or .Net / Mono library, or going for full automation >>>> (I'd suggest MS Office over OpenOffice if going that route). >>>> >>>> >>>> Cheers, >>>> >>>> Rasjid. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On 4/02/2014 2:29 PM, Javier Candeira wrote: >>>>> >>>>> At work I need to manipulate a series of excel spreasheets, and I seem >>>>> to remember that people have discussed the topic in a meeting. I have >>>>> two questions to anyone who's already had to do the same: >>>>> >>>>> - What's the current best Python library for excel manipulation? >>>>> - Are there any limitations for handling Excel sheets under Linux? >>>>> What kind of thing would I need Windows (and presumably, a live copy >>>>> of Office) for? >>>>> >>>>> As background for the question, I'm trying to avoid using Windows at >>>>> all. My ideal workflow would involve users downloading generated >>>>> spreadsheets from a webserver. >>>>> >>>>> Thanks, >>>>> >>>>> Javier >>>>> _______________________________________________ >>>>> >>>>> melbourne-pug mailing list >>>>> melbourne-pug at python.org >>>>> https://mail.python.org/mailman/listinfo/melbourne-pug >>>> >>>> >>>> _______________________________________________ >>>> >>>> melbourne-pug mailing list >>>> melbourne-pug at python.org >>>> https://mail.python.org/mailman/listinfo/melbourne-pug >>> _______________________________________________ >>> melbourne-pug mailing list >>> melbourne-pug at python.org >>> https://mail.python.org/mailman/listinfo/melbourne-pug >> >> >> _______________________________________________ >> melbourne-pug mailing list >> melbourne-pug at python.org >> https://mail.python.org/mailman/listinfo/melbourne-pug >> > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From bianca.rachel.gibson at gmail.com Tue Feb 4 06:41:14 2014 From: bianca.rachel.gibson at gmail.com (Bianca Gibson) Date: Tue, 4 Feb 2014 16:41:14 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: <52F06AFC.2080201@openminddev.net> References: <52F06AFC.2080201@openminddev.net> Message-ID: On 4 February 2014 15:22, Rasjid Wilcox wrote: > Under Linux etc. > Use a head-less copy of OpenOffice, and use its automation (pyUno) to > generate a spreadsheet that way. Be prepared for a lot of pain. > Last time I heard (early 2013) the headless build only worked on Linux. I expect could use its automation with a head on other platforms, but that's gonna be a big binary. Don't know the big binary would be an issue for you. Bianca -------------- next part -------------- An HTML attachment was scrubbed... URL: From javier at candeira.com Tue Feb 4 06:39:13 2014 From: javier at candeira.com (Javier Candeira) Date: Tue, 4 Feb 2014 16:39:13 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: <52F06AFC.2080201@openminddev.net> Message-ID: Yes, I have now received the template sheets, and they are .xlsx, not xls. I'm doing a round of trying different Python libraries, see what happens. Thanks again, everyone. You are all awesome! J On Tue, Feb 4, 2014 at 4:13 PM, Henry Walshaw wrote: > Javier, > > The only thing I could think of trying would be pyodbc (assuing you're on a Windows machine, which I know you were trying to avoid) and trying to write through an ODBC connection. Something like: > > cnxn=pyodbc.connect("""Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)}; Dbq=c:\\datafiles\\XLSfile.xls; readonly=0""", autocommit=True) > > I haven't tested this though, so it's really just clutching at straws. Otherwise I'm not sure of any Python files that modify Excel in place. Good luck! > > Cheers, > Henry > > On 4 Feb 2014, at 3:53 pm, Javier Candeira wrote: > >> Then, just to be clear: there doesn't seem to exist a Python library >> that will roundtrip an excel sheet to its original state just by >> reading it in and writing it again, without modification. Correct? >> >> J >> >> On Tue, Feb 4, 2014 at 3:48 PM, Henry Walshaw wrote: >>> Hi Javier, >>> >>> Like Matt below I'd recommend taking a look at openpyxl (http://pythonhosted.org/openpyxl/) We've used it a lot recently and it does give a great deal over formatting of the outgoing Excel workbook. Unfortunately it does not support reading images and charts from an existing workbook, so they would be lost if you tried reading an existing workbook as a template (openpyxl works with Excel data in memory and only accesses the file handler on save or load, which means that modifying an existing workbook is a case of reading the data in, updating the data and then saving a new excel file over the original one). However I think it's a good choice if all you're doing is manipulating workbooks full of data that the template is applied to later. >>> >>> Cheers >>> -Henry >>> >>> >>> On 4 Feb 2014, at 3:38 pm, Javier Candeira wrote: >>> >>>> Thanks everyone. Some more background and answers: >>>> >>>> - I'm a LibreOffice fan (this is me after a love night with two >>>> LibreOffice hackers [1]), but this has to be MS Office. I'm automating >>>> an entrenched reporting process that produces pixel-perfect A4 >>>> documents for printing, and I can't recreate the accumulated work >>>> expressed in the existing sheets. >>>> - I'm also a Google Spreadsheets fan, to the point where I have >>>> contributed to a library for accessing it. [2] But same as above. >>>> - xlwt or xlutil might work if it allows me to manipulate existing >>>> spreadsheets. If not, I'd try creating my reporting template and >>>> linking it to sheets with sample data, and to recreate the >>>> data-bearing sheets every week/month/whenever. >>>> - I don't really need to generate macros or charts, because there are >>>> templates that have them already. I really only have to put in data in >>>> an existing database (and munge the incoming data a bit before writing >>>> the neat rows and columns). >>>> >>>> [1] http://www.flickr.com/photos/hiperactivo/4364689348/ >>>> [2] https://github.com/burnash/gspread >>>> >>>> Thanks again, and sorry for not giving enough background at the start >>>> of the thread. >>>> >>>> J >>>> >>>> On Tue, Feb 4, 2014 at 3:22 PM, Rasjid Wilcox wrote: >>>>> Hi Javier, >>>>> >>>>> I use xlwt (https://pypi.python.org/pypi/xlwt) to generate XLS spreadsheets. >>>>> It is a fork of pyExcelerator which is not longer maintained. >>>>> >>>>> For fairly standard spreadsheets it works well. You can do basic >>>>> formatting, both in terms of fonts (bold etc), and also in terms of numeric >>>>> formatting (currency and date formats etc). >>>>> >>>>> Last I checked (and I don't think this has changed), it does not do charts >>>>> or fancy stuff like that. >>>>> >>>>> As long as it meets your needs it terms of spreadsheet complexity, it will >>>>> work well. Runs on any platform (will be fine on a Linux web-server), and >>>>> is fast. >>>>> >>>>> I looked into this extensively a few years ago - at the time it was >>>>> definitely the way to go as long as it meets your requirements in terms of >>>>> capabilities. >>>>> >>>>> >>>>> Your other options are: >>>>> >>>>> Under Linux etc. >>>>> Use a head-less copy of OpenOffice, and use its automation (pyUno) to >>>>> generate a spreadsheet that way. Be prepared for a lot of pain. >>>>> Use Jython (or bridge to java in some way) and use one of the Java >>>>> libraries to generate XLS files. At least one of these has basic support >>>>> for charts (although you may need a base template file to work from). >>>>> >>>>> I know there are some .Net libraries that can also generate xlsx files - >>>>> these may work under Mono and IronPython. >>>>> >>>>> >>>>> Under Windows: >>>>> All of the above, with the bonus that any .Net libraries are definitely >>>>> going to work. >>>>> Automate MS Office using Python's COM support. >>>>> >>>>> >>>>> The pro's of using automation (either OpenOffice or MS Office) is that you >>>>> will get access to a much greater range of features. It will, however, be >>>>> slow, and can take noticeable amounts of time to generate a large >>>>> spreadsheet. >>>>> >>>>> Any of the libraries that generate the file directly will be *much* faster, >>>>> and a better fit for server-side deployment. >>>>> >>>>> >>>>> >>>>> Hmm... just seen your last email regarding macros. Not sure if xlwt >>>>> supports that. You may have luck with the xlutils library mentioned - >>>>> otherwise you may need to move to one of the more heavy duty options - >>>>> either using a Java or .Net / Mono library, or going for full automation >>>>> (I'd suggest MS Office over OpenOffice if going that route). >>>>> >>>>> >>>>> Cheers, >>>>> >>>>> Rasjid. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 4/02/2014 2:29 PM, Javier Candeira wrote: >>>>>> >>>>>> At work I need to manipulate a series of excel spreasheets, and I seem >>>>>> to remember that people have discussed the topic in a meeting. I have >>>>>> two questions to anyone who's already had to do the same: >>>>>> >>>>>> - What's the current best Python library for excel manipulation? >>>>>> - Are there any limitations for handling Excel sheets under Linux? >>>>>> What kind of thing would I need Windows (and presumably, a live copy >>>>>> of Office) for? >>>>>> >>>>>> As background for the question, I'm trying to avoid using Windows at >>>>>> all. My ideal workflow would involve users downloading generated >>>>>> spreadsheets from a webserver. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Javier >>>>>> _______________________________________________ >>>>>> >>>>>> melbourne-pug mailing list >>>>>> melbourne-pug at python.org >>>>>> https://mail.python.org/mailman/listinfo/melbourne-pug >>>>> >>>>> >>>>> _______________________________________________ >>>>> >>>>> melbourne-pug mailing list >>>>> melbourne-pug at python.org >>>>> https://mail.python.org/mailman/listinfo/melbourne-pug >>>> _______________________________________________ >>>> melbourne-pug mailing list >>>> melbourne-pug at python.org >>>> https://mail.python.org/mailman/listinfo/melbourne-pug >>> >>> >>> _______________________________________________ >>> melbourne-pug mailing list >>> melbourne-pug at python.org >>> https://mail.python.org/mailman/listinfo/melbourne-pug >>> >> _______________________________________________ >> melbourne-pug mailing list >> melbourne-pug at python.org >> https://mail.python.org/mailman/listinfo/melbourne-pug > > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > From brian at microcomaustralia.com.au Tue Feb 4 22:57:39 2014 From: brian at microcomaustralia.com.au (Brian May) Date: Wed, 5 Feb 2014 08:57:39 +1100 Subject: [melbourne-pug] python future debian In-Reply-To: References: Message-ID: On 4 February 2014 08:57, Brian May wrote: > After last night's talk on Python future, I decided to look up the status > of packages for Debian. > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=736523 > > Oops. Seems to have deteriorated into a silly argument over what teamwork > means :-( > Looks like *everyone*, including myself, got a bit confused. This isn't python-future they are discussing, this is python-concurrency.futures. http://docs.python.org/dev/library/concurrent.futures.html vs http://python-future.org/ It looks like the discussion with concurrent.futures continued on the debian-python mailing list, and is now resolved. Discussion starts at https://lists.debian.org/debian-python/2014/01/msg00044.html -- Brian May -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Tue Feb 4 23:19:01 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 05 Feb 2014 09:19:01 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? References: Message-ID: <85y51qed2h.fsf@benfinney.id.au> Javier Candeira writes: > At work I need to manipulate a series of excel spreasheets What manipulations are needed? > - What's the current best Python library for excel manipulation? I don't know much about them, but Pandas depends on these two: for reading for writing > - Are there any limitations for handling Excel sheets under Linux? Both the above are pure Python and AFAIK don't depend on any Microsoft products for any functionality. -- \ ?There's a certain part of the contented majority who love | `\ anybody who is worth a billion dollars.? ?John Kenneth | _o__) Galbraith, 1992-05-23 | Ben Finney From javier at candeira.com Tue Feb 4 23:38:32 2014 From: javier at candeira.com (Javier Candeira) Date: Wed, 5 Feb 2014 09:38:32 +1100 Subject: [melbourne-pug] python future debian In-Reply-To: References: Message-ID: Reported: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=736523#65 J On Wed, Feb 5, 2014 at 8:57 AM, Brian May wrote: > On 4 February 2014 08:57, Brian May wrote: >> >> After last night's talk on Python future, I decided to look up the status >> of packages for Debian. >> >> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=736523 >> >> Oops. Seems to have deteriorated into a silly argument over what teamwork >> means :-( > > > Looks like *everyone*, including myself, got a bit confused. > > This isn't python-future they are discussing, this is > python-concurrency.futures. > > http://docs.python.org/dev/library/concurrent.futures.html > > vs > > http://python-future.org/ > > It looks like the discussion with concurrent.futures continued on the > debian-python mailing list, and is now resolved. > > Discussion starts at > https://lists.debian.org/debian-python/2014/01/msg00044.html > -- > Brian May > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > From sam at nipl.net Wed Feb 5 17:11:37 2014 From: sam at nipl.net (Sam Watkins) Date: Thu, 6 Feb 2014 03:11:37 +1100 Subject: [melbourne-pug] Excel spreadsheet munging under Linux. Limits? In-Reply-To: References: <52F06AFC.2080201@openminddev.net> Message-ID: <20140205161137.GJ28286@opal.nipl.net> On Tue, Feb 04, 2014 at 04:39:13PM +1100, Javier Candeira wrote: > Yes, I have now received the template sheets, and they are .xlsx, not > xls. I'm doing a round of trying different Python libraries, see what > happens. I'm fairly sure there are public python libraries that can preserve an xlsx template and only change the data. I have looked into it. However, I am developing a program called xlsx_format that does this. The xlsx format is a fairly simple zip of XML files. I previously helped build an 'xls_format' at work using perl. I was attempting to program xlsx_format as a collection of tools tied together in a shell script (as an experiment in "flow based programming") but I did not complete that yet. I can instead do it in python as a libary (and tool), if that's useful. So far, I have been working on xlsx_format in my spare time and intend to release it to the public domain or CC0, so you can use it however you like and we can also use it at work. I am thinking that it should take a collection of 'tables' as input. I'm not sure the best way to represent them, perhaps as an iterable of dicts. Please let me know what you think, and if you have any other requirements. Sam From javier at candeira.com Thu Feb 6 01:24:36 2014 From: javier at candeira.com (Javier Candeira) Date: Thu, 6 Feb 2014 11:24:36 +1100 Subject: [melbourne-pug] Presentations for the March session... Message-ID: Hi everyone, First of all, February: Thanks to everybody who came last Monday. It was one of our most populous sessions lately, and the discussions were great. Extra thanks to all the presenters, and especially to Andrew Walker, who not only sacrificed his own talk in the interest of adhering to the schedule (he'll present in March), he also shouted us some beer! Now, March! - We have Andrew's talk, - I'd like to give a short presentation on how I took the githubpy library I talked about some months ago and made it into an API library for Insightly with about 5 modifications (yay dinamic REST API libraries!). I've also thought a bit about the downsides and tradeoffs of the approach, and will share. - We have an empty slot for a 25 minute talk. It could be you! It could be you, J and the MPUG Organisers From miked at dewhirst.com.au Fri Feb 7 08:21:24 2014 From: miked at dewhirst.com.au (Mike Dewhirst) Date: Fri, 07 Feb 2014 18:21:24 +1100 Subject: [melbourne-pug] foo is None bar is 2 so is foo < bar Message-ID: <52F48974.5030500@dewhirst.com.au> It is in Python 2.7 ... Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> foo = None >>> bar = 2 >>> foo < bar True >>> But it seems not in Python 3.3 ... Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> foo = None >>> bar = 2 >>> foo < bar Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: NoneType() < int() >>> Should I rtfm? Maybe there is a trick to it? Thanks Mike From ben+python at benfinney.id.au Fri Feb 7 08:32:58 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 07 Feb 2014 18:32:58 +1100 Subject: [melbourne-pug] foo is None bar is 2 so is foo < bar References: <52F48974.5030500@dewhirst.com.au> Message-ID: <85wqh7cr85.fsf@benfinney.id.au> Mike Dewhirst writes: > But it seems not in Python 3.3 ... > > Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 > 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> foo = None > >>> bar = 2 > >>> foo < bar > Traceback (most recent call last): > File "", line 1, in > TypeError: unorderable types: NoneType() < int() > >>> > > Should I rtfm? The answer to that question is always ?yes? :-) It's good to understand what is done (read the manual) and why it's done (read the ?What's New? documents). > Maybe there is a trick to it? A trick to what? What are you expecting to happen, and how does this behaviour violate your expectation? -- \ ?We are not gonna be great; we are not gonna be amazing; we are | `\ gonna be *amazingly* amazing!? ?Zaphod Beeblebrox, _The | _o__) Hitch-Hiker's Guide To The Galaxy_, Douglas Adams | Ben Finney From cjlmac at gmail.com Fri Feb 7 08:36:12 2014 From: cjlmac at gmail.com (Chris Maclachlan) Date: Fri, 7 Feb 2014 18:36:12 +1100 Subject: [melbourne-pug] foo is None bar is 2 so is foo < bar In-Reply-To: <52F48974.5030500@dewhirst.com.au> References: <52F48974.5030500@dewhirst.com.au> Message-ID: Hi Mike None is not the same as int(0) - it's a bit like javascript's NaN (not-a-number); it's not 0, it's a complete absence of value; an indeterminate value. Because of this, it can't actually be successfully cast to a numeric: >>> a = None >>> int(a) Traceback (most recent call last): File "", line 1, in TypeError: int() argument must be a string or a number, not 'NoneType' Python 3's a bit stricter about trying to compare incomparable types (See e.g. http://stackoverflow.com/questions/8961005/comparing-none-with-built-in-types-using-arithmetic-operatorsfor more). The way to correctly compare to None is to use "is", ie: >>> a = None >>> b = 1 >>> if a is not b: ... print("woo") ... woo Cheers Chris On Fri, Feb 7, 2014 at 6:21 PM, Mike Dewhirst wrote: > It is in Python 2.7 ... > > Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] > on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> foo = None > >>> bar = 2 > >>> foo < bar > True > >>> > > But it seems not in Python 3.3 ... > > Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 32 > bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> foo = None > >>> bar = 2 > >>> foo < bar > Traceback (most recent call last): > File "", line 1, in > TypeError: unorderable types: NoneType() < int() > >>> > > Should I rtfm? > > Maybe there is a trick to it? > > Thanks > > Mike > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > -------------- next part -------------- An HTML attachment was scrubbed... URL: From miked at dewhirst.com.au Fri Feb 7 09:04:09 2014 From: miked at dewhirst.com.au (Mike Dewhirst) Date: Fri, 07 Feb 2014 19:04:09 +1100 Subject: [melbourne-pug] foo is None bar is 2 so is foo < bar In-Reply-To: References: <52F48974.5030500@dewhirst.com.au> Message-ID: <52F49379.6000401@dewhirst.com.au> On 7/02/2014 6:36pm, Chris Maclachlan wrote: > Hi Mike > > None is not the same as int(0) - it's a bit like javascript's NaN > (not-a-number); it's not 0, it's a complete absence of value; an > indeterminate value. Because of this, it can't actually be successfully > cast to a numeric: > > >>> a = None > >>> int(a) > Traceback (most recent call last): > File "", line 1, in > TypeError: int() argument must be a string or a number, not 'NoneType' > > Python 3's a bit stricter about trying to compare incomparable types > (See e.g. > http://stackoverflow.com/questions/8961005/comparing-none-with-built-in-types-using-arithmetic-operators > for more). Thanks Chris (and Ben). I guess that's what they mean by breaking backwards compatibility. I suppose its for my own good. I'll just rewrite all my code. At least it will still work in Python 2 But it doesn't all have to be changed because ... >>> bar = 'x' >>> foo = None >>> if not foo: ... bar = 'y' ... >>> bar 'y' Mike The way to correctly compare to None is to use "is", ie: > > >>> a = None > >>> b = 1 > >>> if a is not b: > ... print("woo") > ... > woo > > Cheers > > Chris > > > On Fri, Feb 7, 2014 at 6:21 PM, Mike Dewhirst > wrote: > > It is in Python 2.7 ... > > Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit > (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> foo = None > >>> bar = 2 > >>> foo < bar > True > >>> > > But it seems not in Python 3.3 ... > > Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC > v.1600 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> foo = None > >>> bar = 2 > >>> foo < bar > Traceback (most recent call last): > File "", line 1, in > TypeError: unorderable types: NoneType() < int() > >>> > > Should I rtfm? > > Maybe there is a trick to it? > > Thanks > > Mike > > _________________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/__mailman/listinfo/melbourne-pug > > > > > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > From sam at nipl.net Fri Feb 7 09:20:56 2014 From: sam at nipl.net (Sam Watkins) Date: Fri, 7 Feb 2014 19:20:56 +1100 Subject: [melbourne-pug] foo is None bar is 2 so is foo < bar In-Reply-To: <52F48974.5030500@dewhirst.com.au> References: <52F48974.5030500@dewhirst.com.au> Message-ID: <20140207082056.GE17867@opal.nipl.net> On Fri, Feb 07, 2014 at 06:21:24PM +1100, Mike Dewhirst wrote: > Maybe there is a trick to it? >>> (None or 0) < 2 True From javier at candeira.com Fri Feb 14 05:01:18 2014 From: javier at candeira.com (Javier Candeira) Date: Fri, 14 Feb 2014 15:01:18 +1100 Subject: [melbourne-pug] Talks for March Message-ID: We already have two talks for our March meeting. There is one open spot, though. Does anyone have anything they'd like to talk about? J From crolfe at gmail.com Fri Feb 14 08:27:30 2014 From: crolfe at gmail.com (Colin Rolfe) Date: Fri, 14 Feb 2014 18:27:30 +1100 Subject: [melbourne-pug] Job opportunity for Python/Django developers Message-ID: <52FDC562.8020604@gmail.com> The company I work for is growing and looking for an additional developer: http://www.seek.com.au/job/26012553 Feel free to let me know (or apply directly) if you are interested or have any questions about the role. Bonus points if you've been to MelbDjango before! -Colin From javier at candeira.com Fri Feb 14 22:29:29 2014 From: javier at candeira.com (Javier Candeira) Date: Sat, 15 Feb 2014 08:29:29 +1100 Subject: [melbourne-pug] Paging Jason King Message-ID: I've lost Jason King's email address, so the list it is. Jason, can you please email me privately? Thanks, Javier From david.crisp at gmail.com Thu Feb 20 23:31:25 2014 From: david.crisp at gmail.com (David Crisp) Date: Fri, 21 Feb 2014 09:31:25 +1100 Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question Message-ID: The following question is more along the lines of "good practice" rather than "how do you do it" . If I have a name:value list of values that I want to read into a dict for ease of lookup, lets define them as: name | value ========== ItemOne : 10 ItemOne : 10 ItemOne : 10 ItemOne : 10 ItemTwo : 20 ItemTwo : 20 ItemTwo : 20 ItemTwo : 20 ItemThree : 30 ItemThree : 30 ItemThree : 30 ItemThree : 30 Now, obviously there are duplicates in that list. If I use a simple loop such as the following: for each_name in list: item[name] = value Then I will get a dict with three pairs in it: ItemOne : 10 ItemTwo : 20 ItemThree : 30 Which is what I want. Now, MY question is, is there any harm in creating the dict that way and looping through all those values multiple times and re-defining the values constantly (to the same thing). OR, should I put a check in there such as: if name not in item: # add name and its pair value to dict In my case I HAVE added checking in but I was wondering if it was really needed. Given no matter what, in either case, the resulting dict would be the same. Regards, David -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthony.briggs at gmail.com Thu Feb 20 23:39:20 2014 From: anthony.briggs at gmail.com (Anthony Briggs) Date: Fri, 21 Feb 2014 09:39:20 +1100 Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question In-Reply-To: References: Message-ID: Sounds like you don't need the checking, since the items are all identical, *but* if the values differ for a particular key, then the behaviour will be different (earliest with checking, latest without). It you have itemOne: 10, then itemOne: 20, the checked version will have 10 for itemOne You can also use the dict() function or dictionary comprehensions to create your dictionary: item = dict( (key, value) for key, value in list ) or item = {key: value for (key, value) in list} Finally, I wouldn't go overriding the builtin list() function with your variable name though. Call it something less generic, like item_pricing. Hope that helps, Anthony On 21 February 2014 09:31, David Crisp wrote: > The following question is more along the lines of "good practice" rather > than "how do you do it" . > > If I have a name:value list of values that I want to read into a dict for > ease of lookup, lets define them as: > > name | value > ========== > ItemOne : 10 > ItemOne : 10 > ItemOne : 10 > ItemOne : 10 > ItemTwo : 20 > ItemTwo : 20 > ItemTwo : 20 > ItemTwo : 20 > ItemThree : 30 > ItemThree : 30 > ItemThree : 30 > ItemThree : 30 > > Now, obviously there are duplicates in that list. If I use a simple > loop such as the following: > for each_name in list: > item[name] = value > > Then I will get a dict with three pairs in it: > ItemOne : 10 > ItemTwo : 20 > ItemThree : 30 > > Which is what I want. > > Now, MY question is, is there any harm in creating the dict that way > and looping through all those values multiple times and re-defining the > values constantly (to the same thing). OR, should I put a check in there > such as: > > if name not in item: > # add name and its pair value to dict > > > In my case I HAVE added checking in but I was wondering if it was really > needed. Given no matter what, in either case, the resulting dict would be > the same. > > Regards, > David > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From william.leslie.ttg at gmail.com Thu Feb 20 23:45:59 2014 From: william.leslie.ttg at gmail.com (William ML Leslie) Date: Fri, 21 Feb 2014 09:45:59 +1100 Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question In-Reply-To: References: Message-ID: On 21/02/2014 9:40 am, "Anthony Briggs" wrote: > > You can also use the dict() function or dictionary comprehensions to create your dictionary: > item = dict( (key, value) for key, value in list ) > Otherwise written: item = dict(list) -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthony.briggs at gmail.com Thu Feb 20 23:52:21 2014 From: anthony.briggs at gmail.com (Anthony Briggs) Date: Fri, 21 Feb 2014 09:52:21 +1100 Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question In-Reply-To: References: Message-ID: Yep, just realised that - if it's stored as a list of key, value pairs. On 21 February 2014 09:45, William ML Leslie wrote: > > On 21/02/2014 9:40 am, "Anthony Briggs" wrote: > > > > You can also use the dict() function or dictionary comprehensions to > create your dictionary: > > item = dict( (key, value) for key, value in list ) > > > > Otherwise written: > item = dict(list) > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joshbode at gmail.com Thu Feb 20 23:49:12 2014 From: joshbode at gmail.com (Josh Bode) Date: Fri, 21 Feb 2014 09:49:12 +1100 Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question In-Reply-To: References: Message-ID: Hi David, On my machine, on a simple, pathological example, it is significantly faster (around a factor of 2) to check for duplicates. However, it depends if you expect to have many duplicates. In the second example with only one duplicated item, it is faster to not check. The winner in both cases, however, is to use dict() directly. Cheers, Josh IPython session: In [1]: l = [('a', 1)] * 10000 In [2]: def f(l): ...: d = {} ...: for k, v in l: ...: d[k] = v ...: return d ...: In [3]: def g(l): ...: d = {} ...: for k, v in l: ...: if k not in d: ...: d[k] = v ...: return d ...: In [4]: %timeit f(l) 1000 loops, best of 3: 1.34 ms per loop In [5]: %timeit g(l) 100 loops, best of 3: 773 ?s per loop In [6]: %timeit dict(l) 1000 loops, best of 3: 371 ?s per loop In [7]: m = [(str(x), x) for x in range(10000)] + [('0', 0)] In [8]: %timeit f(m) 1000 loops, best of 3: 1.99 ms per loop In [9]: %timeit g(m) 100 loops, best of 3: 2.48 ms per loop In [10]: %timeit dict(m) 100 loops, best of 3: 943 ?s per loop On 21 February 2014 09:31, David Crisp wrote: > The following question is more along the lines of "good practice" rather > than "how do you do it" . > > If I have a name:value list of values that I want to read into a dict for > ease of lookup, lets define them as: > > name | value > ========== > ItemOne : 10 > ItemOne : 10 > ItemOne : 10 > ItemOne : 10 > ItemTwo : 20 > ItemTwo : 20 > ItemTwo : 20 > ItemTwo : 20 > ItemThree : 30 > ItemThree : 30 > ItemThree : 30 > ItemThree : 30 > > Now, obviously there are duplicates in that list. If I use a simple > loop such as the following: > for each_name in list: > item[name] = value > > Then I will get a dict with three pairs in it: > ItemOne : 10 > ItemTwo : 20 > ItemThree : 30 > > Which is what I want. > > Now, MY question is, is there any harm in creating the dict that way > and looping through all those values multiple times and re-defining the > values constantly (to the same thing). OR, should I put a check in there > such as: > > if name not in item: > # add name and its pair value to dict > > > In my case I HAVE added checking in but I was wondering if it was really > needed. Given no matter what, in either case, the resulting dict would be > the same. > > Regards, > David > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Thu Feb 20 23:56:22 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 21 Feb 2014 09:56:22 +1100 Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question References: Message-ID: <85ob212y21.fsf@benfinney.id.au> David Crisp writes: > name | value > ========== > ItemOne : 10 > ItemOne : 10 > ItemOne : 10 > ItemOne : 10 > ItemTwo : 20 > ItemTwo : 20 > ItemTwo : 20 > ItemTwo : 20 > ItemThree : 30 > ItemThree : 30 > ItemThree : 30 > ItemThree : 30 If you're confident there will frequently be duplicated lines, and you want to ignore the duplicates, I'd recommend (on Unix) filtering the list to remove them:: $ cat items | sort | uniq > items_dedup Then you can read the ?items_dedup? file in your Python program. You can even write your Python program as a filter (read the input lines from ?sys.stdin?, write the result to ?sys.stdout?) and just hook it into that command pipeline. If the program you're writing is named ?do_more_processing?:: $ cat items | sort | uniq | do_more_processing > outputfile -- \ ?Science is a way of trying not to fool yourself. The first | `\ principle is that you must not fool yourself, and you are the | _o__) easiest person to fool.? ?Richard P. Feynman, 1964 | Ben Finney From dcrisp at netspace.net.au Fri Feb 21 00:55:10 2014 From: dcrisp at netspace.net.au (David Crisp) Date: Fri, 21 Feb 2014 10:55:10 +1100 (EST) Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question In-Reply-To: References: Message-ID: Thanks for the thoughts! In this case I am reading in data from a SQL database query(pymssql) and feeding it into an excel spreedsheet (xlwt) Line by line which means I have to assemble the dict line by line. And yes, the name i chose in the email to represent what I was doing was generic. THe one I am actually using is representative of the data its holding! :) as apposed to a generic word which could actually be a keyword! And Yes yes, Theres quite a lot of data. Many of these query returns have 3000 rows of data! Regards, David On Fri, 21 Feb 2014, William ML Leslie wrote: > On 21/02/2014 9:40 am, "Anthony Briggs" wrote: >> >> You can also use the dict() function or dictionary comprehensions to > create your dictionary: >> item = dict( (key, value) for key, value in list ) >> > > Otherwise written: > item = dict(list) > From joshbode at gmail.com Fri Feb 21 01:03:31 2014 From: joshbode at gmail.com (Josh Bode) Date: Fri, 21 Feb 2014 11:03:31 +1100 Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question In-Reply-To: References: Message-ID: If the data is in a database, and the rows are truly duplicates (i.e. keys always map to the same values) you could probably remove the duplicates via the query before loading the data into Python. The DISTINCT keyword will do this for you, i.e. SELECT DISTINCT * FROM x; Otherwise, a GROUP BY could achieve the same. On 21 February 2014 10:55, David Crisp wrote: > Thanks for the thoughts! > > In this case I am reading in data from a SQL database query(pymssql) and > feeding it into an excel spreedsheet (xlwt) Line by line which means I have > to assemble the dict line by line. > > And yes, the name i chose in the email to represent what I was doing was > generic. THe one I am actually using is representative of the data its > holding! :) as apposed to a generic word which could actually be a > keyword! > > And Yes yes, Theres quite a lot of data. Many of these query returns > have 3000 rows of data! > > Regards, > David > > > On Fri, 21 Feb 2014, William ML Leslie wrote: > > On 21/02/2014 9:40 am, "Anthony Briggs" wrote: >> >>> >>> You can also use the dict() function or dictionary comprehensions to >>> >> create your dictionary: >> >>> item = dict( (key, value) for key, value in list ) >>> >>> >> Otherwise written: >> item = dict(list) >> >> _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Fri Feb 21 01:15:25 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 21 Feb 2014 11:15:25 +1100 Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question References: Message-ID: <85k3cp2uea.fsf@benfinney.id.au> David Crisp writes: > In this case I am reading in data from a SQL database query(pymssql) If you want to eliminate duplicates from your query, do so with ?SELECT DISTINCT?. Then you don't ever get the duplicate rows in the first place :-) -- \ ?Ocean, n. A body of water occupying about two-thirds of a | `\ world made for man ? who has no gills.? ?Ambrose Bierce, _The | _o__) Devil's Dictionary_, 1906 | Ben Finney From dcrisp at netspace.net.au Fri Feb 21 01:30:16 2014 From: dcrisp at netspace.net.au (David Crisp) Date: Fri, 21 Feb 2014 11:30:16 +1100 (EST) Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question In-Reply-To: <85k3cp2uea.fsf@benfinney.id.au> References: <85k3cp2uea.fsf@benfinney.id.au> Message-ID: Ahh :) I also meant to say that whilest the particular fields I am talking about in this question are the same, data at the end of those rows is different, and will be treated differently. I really abstracted my query away from what I am actually trying to do exactly and more along the lines of how effiecient or ineficient I am doing things. This particular SQL query was originally - before I optimised it for the person here working with it - returning upwards of 100,000 rows of data because it had been produced by sombody using the built in data join wizard in MSSQL Server Management Studio. They would then post process it in code (various flavours of .net and C ) and come up with the results they were wanting... The query I hand cut for them and tested now returns exactly the data they wish in just enough rows to contain it all. No need to post process it as such. Regards, David On Fri, 21 Feb 2014, Ben Finney wrote: > David Crisp writes: > >> In this case I am reading in data from a SQL database query(pymssql) > > If you want to eliminate duplicates from your query, do so with ?SELECT > DISTINCT?. Then you don't ever get the duplicate rows in the first place :-) > > -- > \ ?Ocean, n. A body of water occupying about two-thirds of a | > `\ world made for man ? who has no gills.? ?Ambrose Bierce, _The | > _o__) Devil's Dictionary_, 1906 | > Ben Finney > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug From joshbode at gmail.com Fri Feb 21 01:40:07 2014 From: joshbode at gmail.com (Josh Bode) Date: Fri, 21 Feb 2014 11:40:07 +1100 Subject: [melbourne-pug] Question about adding members to list : hypothetical just for interest kind of question In-Reply-To: References: <85k3cp2uea.fsf@benfinney.id.au> Message-ID: If the data you need to collapse is being duplicated across multiple rows due to distinct values in other columns, you could generate that data in a separate, more focussed query. That way you would avoid having to de-duplicate that information, but you do have an additional query to maintain and fetch data from, though. At the data volumes you're talking about this might not be an issue, but for larger data-sets this would be more efficient. On 21 February 2014 11:30, David Crisp wrote: > Ahh :) I also meant to say that whilest the particular fields I am > talking about in this question are the same, data at the end of those rows > is different, and will be treated differently. > > I really abstracted my query away from what I am actually trying to do > exactly and more along the lines of how effiecient or ineficient I am doing > things. > > This particular SQL query was originally - before I optimised it for the > person here working with it - returning upwards of 100,000 rows of data > because it had been produced by sombody using the built in data join wizard > in MSSQL Server Management Studio. They would then post process it in code > (various flavours of .net and C ) and come up with the results they were > wanting... The query I hand cut for them and tested now returns exactly > the data they wish in just enough rows to contain it all. No need to post > process it as such. > > Regards, > David > > > > > On Fri, 21 Feb 2014, Ben Finney wrote: > > David Crisp writes: >> >> In this case I am reading in data from a SQL database query(pymssql) >>> >> >> If you want to eliminate duplicates from your query, do so with ?SELECT >> DISTINCT?. Then you don't ever get the duplicate rows in the first place >> :-) >> >> -- >> \ ?Ocean, n. A body of water occupying about two-thirds of a | >> `\ world made for man ? who has no gills.? ?Ambrose Bierce, _The | >> _o__) Devil's Dictionary_, 1906 | >> Ben Finney >> >> _______________________________________________ >> melbourne-pug mailing list >> melbourne-pug at python.org >> https://mail.python.org/mailman/listinfo/melbourne-pug >> > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Sat Feb 22 11:51:14 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 22 Feb 2014 10:51:14 -0000 Subject: [melbourne-pug] python GIS volunteers? In-Reply-To: (dan@acommoncreative.com's message of "Mon, 3 Feb 2014 17:16:30 +1100") References: Message-ID: <85ioswfqm4.fsf@benfinney.id.au> dan writes: > In addition to my day job, I am currently devoting my free time to > assisting with the online side of the Public Transport Not Traffic > Campaign (http://www.publictransportnottraffic.org/). > [?] Yes, this campaign aligns well with my values. I would like to volunteer my time and skills to the project. > Thanks for reading, > Dan Peade Could I ask you to configure your mail client to put your correct name in the ?From? field? Currently it just says ?dan? which isn't too helpful in reading a list of messages :-) > If you think you might be interested in getting involved and helping > out, or know anyone who would be, please get in touch! Your skills > would really make a difference to the success of the campaign. I'm interested. What should I know/do/read next? -- \ ?Creativity can be a social contribution, but only in so far as | `\ society is free to use the results.? ?Richard Stallman | _o__) | Ben Finney From ben+python at benfinney.id.au Sat Feb 22 23:40:39 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 23 Feb 2014 09:40:39 +1100 Subject: [melbourne-pug] Delayed posting to this forum (was: python GIS volunteers?) References: <85ioswfqm4.fsf@benfinney.id.au> Message-ID: <857g8m3h5k.fsf_-_@benfinney.id.au> Ben Finney writes: > Yes, this campaign aligns well with my values. I would like to > volunteer my time and skills to the project. Well, that was unexpected. Why the twenty-day-delayed post of that message? I am not aware of anything I did to cause that. -- \ ?By instructing students how to learn, unlearn, and relearn, a | `\ powerful new dimension can be added to education.? ?Alvin | _o__) Toffler, _Future Shock_, 1970 | Ben Finney From noonslists at gmail.com Sun Feb 23 01:45:26 2014 From: noonslists at gmail.com (Noon Silk) Date: Sun, 23 Feb 2014 11:45:26 +1100 Subject: [melbourne-pug] Delayed posting to this forum (was: python GIS volunteers?) In-Reply-To: <857g8m3h5k.fsf_-_@benfinney.id.au> References: <85ioswfqm4.fsf@benfinney.id.au> <857g8m3h5k.fsf_-_@benfinney.id.au> Message-ID: On Sun, Feb 23, 2014 at 9:40 AM, Ben Finney wrote: > Ben Finney writes: > > > Yes, this campaign aligns well with my values. I would like to > > volunteer my time and skills to the project. > > Well, that was unexpected. Why the twenty-day-delayed post of that > message? I am not aware of anything I did to cause that. > http://www.quickmeme.com/img/67/6797e3257bc99ba399af8c08baf0db6964328d733f726ed2acead5f849138cb1.jpg Received: from jigong.madmonks.org (localhost [127.0.0.1]) by jigong.madmonks.org (Postfix) with ESMTP id 5DE65DC02A; Sat, 22 Feb 2014 21:43:24 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on vmx15867.hosting24.com.au X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=NO_RELAYS autolearn=ham version=3.3.2 Received: by jigong.madmonks.org (Postfix, from userid 1002) id EF14BDC027; Mon, 3 Feb 2014 21:16:35 +1100 (EST) Looks like it hung out on that server for questioning. > > -- > \ "By instructing students how to learn, unlearn, and relearn, a | > `\ powerful new dimension can be added to education." --Alvin | > _o__) Toffler, _Future Shock_, 1970 | > Ben Finney > > _______________________________________________ > melbourne-pug mailing list > melbourne-pug at python.org > https://mail.python.org/mailman/listinfo/melbourne-pug > -- Noon Silk Fancy a quantum lunch? https://sites.google.com/site/quantumlunch/ "Every morning when I wake up, I experience an exquisite joy -- the joy of being this signature." -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Sun Feb 23 01:49:57 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 23 Feb 2014 11:49:57 +1100 Subject: [melbourne-pug] Delayed posting to this forum References: <85ioswfqm4.fsf@benfinney.id.au> <857g8m3h5k.fsf_-_@benfinney.id.au> Message-ID: <8538ja3b62.fsf@benfinney.id.au> Noon Silk writes: > Looks like it hung out on that server for questioning. Thanks, Noon. Okay, I'll investigate further with that provider. Sorry for the noise, folks. -- \ ?Corporation, n. An ingenious device for obtaining individual | `\ profit without individual responsibility.? ?Ambrose Bierce, | _o__) _The Devil's Dictionary_, 1906 | Ben Finney From gus at projectgus.com Sun Feb 23 01:46:43 2014 From: gus at projectgus.com (Angus Gratton) Date: Sun, 23 Feb 2014 11:46:43 +1100 Subject: [melbourne-pug] Delayed posting to this forum (was: python GIS volunteers?) In-Reply-To: <857g8m3h5k.fsf_-_@benfinney.id.au> References: <85ioswfqm4.fsf@benfinney.id.au> <857g8m3h5k.fsf_-_@benfinney.id.au> Message-ID: <20140223004638.GA15712@ex2.chainxor.org> On Sun, Feb 23, 2014 at 09:40:39AM +1100, Ben Finney wrote: > Well, that was unexpected. Why the twenty-day-delayed post of that > message? I am not aware of anything I did to cause that. Hi Ben, That's kinda strange. The headers show your mail got held for 19 days at jigong.madmonks.org, before moving on to the right places: Received: from localhost (HELO mail.python.org) (127.0.0.1) by albatross.python.org with SMTP; 22 Feb 2014 11:51:14 +0100 Received: from jigong.madmonks.org (unknown [125.214.77.42]) by mail.python.org (Postfix) with ESMTP for ; Sat, 22 Feb 2014 11:51:13 +0100 (CET) Received: from jigong.madmonks.org (localhost [127.0.0.1]) by jigong.madmonks.org (Postfix) with ESMTP id 5DE65DC02A; Sat, 22 Feb 2014 21:43:24 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on vmx15867.hosting24.com.au X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=NO_RELAYS autolearn=ham version=3.3.2 Received: by jigong.madmonks.org (Postfix, from userid 1002) id EF14BDC027; Mon, 3 Feb 2014 21:16:35 +1100 (EST)Received: from localhost (HELO mail.python.org) (127.0.0.1) by albatross.python.org with SMTP; 22 Feb 2014 11:51:14 +0100 Received: from jigong.madmonks.org (unknown [125.214.77.42]) by mail.python.org (Postfix) with ESMTP for ; Sat, 22 Feb 2014 11:51:13 +0100 (CET) Received: from jigong.madmonks.org (localhost [127.0.0.1]) by jigong.madmonks.org (Postfix) with ESMTP id 5DE65DC02A; Sat, 22 Feb 2014 21:43:24 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on vmx15867.hosting24.com.au X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=NO_RELAYS autolearn=ham version=3.3.2 Received: by jigong.madmonks.org (Postfix, from userid 1002) id EF14BDC027; Mon, 3 Feb 2014 21:16:35 +1100 (EST) Is jigong you? The reverse lookup for jigong points back to the hosting24 domain name (also mentioned in the same headers), which seems to be the hostname in the headers for some of your other posts to the list (like the second email.) Grepping my maildir, only the 20-days-delayed email has jigong in the headers. So I'm guessing possibly the mail.python.org server rejected/held it as the sender hostname (jigong) didn't match the IP reverse DNS (hosting24)? Or some other mailserver config shenanigans? - Angus PS I confess to finding the inner workings of email kind of fascinating, probably because I've never had to work with mailservers for an extended period of time. PPS I'm also procrastinating a job I don't want to do. Hard to tell? ;) From javier at candeira.com Mon Feb 24 04:25:17 2014 From: javier at candeira.com (Javier Candeira) Date: Mon, 24 Feb 2014 14:25:17 +1100 Subject: [melbourne-pug] March MPUG Meeting: Timeouts, REST APIs. Monday 3 February, 6PM, Inspire 9, 41 Stewart St Richmond Message-ID: Dear Melbourne Pythonistas, A week from now, Monday 3, we'll hold the March meeting of the Melbourne Python Users Group. Time: 6pm Venue: Inspire 9, 41 Stewart St. Richmond. 50m from Richmond Train Station. Program: * Andrew Walker -- How Hard Could it be to Implement Timeouts? * Javier Candeira -- Implementing a dynamic client for a REST API live (by cheating). As usual, we'll order pizza, with a $10 contribution required. Drinks are BYO, and I'll bring a six-pack that was left over from last week. If you have a tip to share, or want to give a short presentation on a library you've been using lately, please just come forward at the start of the meeting. With much appreciation to Inspire 9 for the donation of the use of the venue, The MPUG organisers From lars at yencken.org Thu Feb 27 06:07:37 2014 From: lars at yencken.org (Lars Yencken) Date: Thu, 27 Feb 2014 16:07:37 +1100 Subject: [melbourne-pug] Job: Data Engineer / Data Scientist at 99designs Message-ID: Hi guys, 99designs, where I work, is hunting for another data person to join our small analytics team. Our Melbourne office is on the floor above Inspire9, where we run our regular MPUG meetups, and is where all our dev work gets done. We're not exactly a Python shop, but this role has plenty of Python in it, since it's such a good language for working with and exploring data. So, I hope you don't mind me posting it here. This role has a backend engineering flavour, but we're certainly open to people with other special skills or experience. If you're interested, please read on: https://99designs.wufoo.com/forms/data-scientistdata-engineer-for-99designs/ Best, Lars -------------- next part -------------- An HTML attachment was scrubbed... URL: From ed at pythoncharmers.com Fri Feb 28 06:03:57 2014 From: ed at pythoncharmers.com (Ed Schofield) Date: Fri, 28 Feb 2014 16:03:57 +1100 Subject: [melbourne-pug] Position for a scientific Python programmer Message-ID: Hi everyone! Is anyone interested in a scientific Python programming job? I have been asked to post the job ad below. It looks like a good opportunity for the right person. If you?re interested, please contact Daehyok Shin directly (details below). Cheers, Ed ??????? Scientific Python Programmer (Melbourne or Canberra, Australia) $76,025 - $86,438 per annum, plus an additional 15.4% superannuation Scientific Python programmer. We are looking for a scientific Python developer to work in the area of hydrological modelling and water forecasting. You will be involved in developing forecasting tools to produce and deliver water information products at a variety of time-scales with an emphasis on web delivery of products. Full-time for one-year. ? professional knowledge and experience of the Python standard library as well as key third party packages for scientific computing such as NumPy, pandas, matplotlib and PyTables ? experience with basic software engineering practices (e.g. distributed VCS, issue tracking, code review) ? experience with shell scripting and basic Linux system administration tasks ? familiarity with scientific data management practices and formats (e.g NetCDF and HDF) and/or relational databases ? web development experience (with Javascript/HTML/CSS) would be an additional benefit You will be part of a team working across a number of projects delivering systems and development support, as such, strong communication skills are a must. Contact: Daehyok Shin, The Bureau of Meteorology, Australia E-mail contact: d.shin at bom.gov.au No telecommuting ??????? -- Dr. Edward Schofield Python Charmers +61 (0)405 676 229 http://pythoncharmers.com -------------- next part -------------- An HTML attachment was scrubbed... URL: