From eliswilson at hushmail.com Wed May 1 01:12:38 2013 From: eliswilson at hushmail.com (eliswilson at hushmail.com) Date: Tue, 30 Apr 2013 19:12:38 -0400 Subject: [python-uk] Biggest Fake Conference in Computer Science Message-ID: <20130430231238.7B5D7E6736@smtp.hushmail.com> Biggest Fake Conference in Computer Science We are researchers from different parts of the world and conducted a study on the world?s biggest bogus computer science conference WORLDCOMP http://sites.google.com/site/worlddump1 organized by Prof. Hamid Arabnia from University of Georgia, USA. We submitted a fake paper to WORLDCOMP 2011 and again (the same paper with a modified title) to WORLDCOMP 2012. This paper had numerous fundamental mistakes. Sample statements from that paper include: (1). Binary logic is fuzzy logic and vice versa (2). Pascal developed fuzzy logic (3). Object oriented languages do not exhibit any polymorphism or inheritance (4). TCP and IP are synonyms and are part of OSI model (5). Distributed systems deal with only one computer (6). Laptop is an example for a super computer (7). Operating system is an example for computer hardware Also, our paper did not express any conceptual meaning. However, it was accepted both the times without any modifications (and without any reviews) and we were invited to submit the final paper and a payment of $500+ fee to present the paper. We decided to use the fee for better purposes than making Prof. Hamid Arabnia richer. After that, we received few reminders from WORLDCOMP to pay the fee but we never responded. This fake paper is different from the two fake papers already published (see https://sites.google.com/site/worlddump4 for details) in WORLDCOMP. We MUST say that you should look at the above website if you have any thoughts of participating in WORLDCOMP. DBLP and other indexing agencies have stopped indexing WORLDCOMP?s proceedings since 2011 due to its fakeness. See http://www.informatik.uni-trier.de/~ley/db/conf/icai/index.html for of one of the conferences of WORLDCOMP and notice that there is no listing after 2010. See Section 2 of http://sites.google.com/site/dumpconf for comments from well-known researchers about WORLDCOMP. The status of your WORLDCOMP papers can be changed from scientific to other (i.e., junk or non-technical) at any time. Better not to have a paper than having it in WORLDCOMP and spoil the resume and peace of mind forever! Our study revealed that WORLDCOMP is money making business, using University of Georgia mask, for Prof. Hamid Arabnia. He is throwing out a small chunk of that money (around 20 dollars per paper published in WORLDCOMP?s proceedings) to his puppet (Mr. Ashu Solo or A.M.G. Solo) who publicizes WORLDCOMP and also defends it at various forums, using fake/anonymous names. The puppet uses fake names and defames other conferences to divert traffic to WORLDCOMP. He also makes anonymous phone calls and threatens the critiques of WORLDCOMP (See Item 7 of Section 5 of above website). That is, the puppet does all his best to get a maximum number of papers published at WORLDCOMP to get more money into his (and Prof. Hamid Arabnia?s) pockets. Prof. Hamid Arabnia makes a lot of tricks. For example, he appeared in a newspaper to fool the public, claiming him a victim of cyber-attack (see Item 8 in Section 5 of above website). Monte Carlo Resort (the venue of WORLDCOMP for more than 10 years, until 2012) has refused to provide the venue for WORLDCOMP?13 because of the fears of their image being tarnished due to WORLDCOMP?s fraudulent activities. That is why WORLDCOMP?13 is taking place at a different resort. WORLDCOMP will not be held after 2013. The draft paper submission deadline is over but still there are no committee members, no reviewers, and there is no conference Chairman. The only contact details available on WORLDCOMP?s website is just an email address! We ask Prof. Hamid Arabnia to publish all reviews for all the papers (after blocking identifiable details) since 2000 conference. Reveal the names and affiliations of all the reviewers (for each year) and how many papers each reviewer had reviewed on average. We also ask him to look at the Open Challenge (Section 6) at https://sites.google.com/site/moneycomp1 and respond if he has any professional values. Sorry for posting to multiple lists. Spreading the word is the only way to stop this bogus conference. Please forward this message to other mailing lists and people. We are shocked with Prof. Hamid Arabnia and his puppet?s activities at http://worldcomp-fake-bogus.blogspot.com Search Google using the keyword worldcomp fake for additional links. From stestagg at gmail.com Wed May 1 01:52:39 2013 From: stestagg at gmail.com (Stestagg) Date: Wed, 1 May 2013 00:52:39 +0100 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: <20130430231238.7B5D7E6736@smtp.hushmail.com> References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: Is this just spam? It certainly has that feel about it.. On Wed, May 1, 2013 at 12:12 AM, wrote: > Biggest Fake Conference in Computer Science > > We are researchers from different parts of the world and conducted a study > on the world?s biggest > bogus computer science conference WORLDCOMP > http://sites.google.com/site/worlddump1 > organized by Prof. Hamid Arabnia from University of Georgia, USA. > > > We submitted a fake paper to WORLDCOMP 2011 and again (the same paper with > a modified title) to > WORLDCOMP 2012. This paper had numerous fundamental mistakes. Sample > statements from that > paper include: > > (1). Binary logic is fuzzy logic and vice versa > (2). Pascal developed fuzzy logic > (3). Object oriented languages do not exhibit any polymorphism or > inheritance > (4). TCP and IP are synonyms and are part of OSI model > (5). Distributed systems deal with only one computer > (6). Laptop is an example for a super computer > (7). Operating system is an example for computer hardware > > > Also, our paper did not express any conceptual meaning. However, it was > accepted both the times > without any modifications (and without any reviews) and we were invited to > submit the final paper > and a payment of $500+ fee to present the paper. We decided to use the fee > for better purposes than > making Prof. Hamid Arabnia richer. After that, we received few reminders > from WORLDCOMP to pay > the fee but we never responded. This fake paper is different from the two > fake papers already published > (see https://sites.google.com/site/worlddump4 for details) in WORLDCOMP. > > > We MUST say that you should look at the above website if you have any > thoughts of participating in > WORLDCOMP. DBLP and other indexing agencies have stopped indexing > WORLDCOMP?s proceedings > since 2011 due to its fakeness. See > http://www.informatik.uni-trier.de/~ley/db/conf/icai/index.html for > of one of the conferences of WORLDCOMP and notice that there is no listing > after 2010. See Section 2 of > http://sites.google.com/site/dumpconf for comments from well-known > researchers about > WORLDCOMP. > > > The status of your WORLDCOMP papers can be changed from scientific to > other (i.e., junk or > non-technical) at any time. Better not to have a paper than having it in > WORLDCOMP and spoil the > resume and peace of mind forever! > > > Our study revealed that WORLDCOMP is money making business, using > University of Georgia mask, for > Prof. Hamid Arabnia. He is throwing out a small chunk of that money > (around 20 dollars per paper > published in WORLDCOMP?s proceedings) to his puppet (Mr. Ashu Solo or > A.M.G. Solo) who publicizes > WORLDCOMP and also defends it at various forums, using fake/anonymous > names. The puppet uses > fake names and defames other conferences to divert traffic to WORLDCOMP. > He also makes anonymous > phone calls and threatens the critiques of WORLDCOMP (See Item 7 of > Section 5 of above website). That > is, the puppet does all his best to get a maximum number of papers > published at WORLDCOMP to get > more money into his (and Prof. Hamid Arabnia?s) pockets. Prof. Hamid > Arabnia makes a lot of tricks. For > example, he appeared in a newspaper to fool the public, claiming him a > victim of cyber-attack (see Item > 8 in Section 5 of above website). > > > Monte Carlo Resort (the venue of WORLDCOMP for more than 10 years, until > 2012) has refused to > provide the venue for WORLDCOMP?13 because of the fears of their image > being tarnished due to > WORLDCOMP?s fraudulent activities. That is why WORLDCOMP?13 is taking > place at a different resort. > WORLDCOMP will not be held after 2013. > > > The draft paper submission deadline is over but still there are no > committee members, no reviewers, > and there is no conference Chairman. The only contact details available on > WORLDCOMP?s website is > just an email address! > > We ask Prof. Hamid Arabnia to publish all reviews for all the papers > (after blocking identifiable details) > since 2000 conference. Reveal the names and affiliations of all the > reviewers (for each year) and how > many papers each reviewer had reviewed on average. We also ask him to look > at the Open Challenge > (Section 6) at https://sites.google.com/site/moneycomp1 and respond if he > has any professional values. > > > Sorry for posting to multiple lists. Spreading the word is the only way to > stop this bogus conference. > Please forward this message to other mailing lists and people. > > > We are shocked with Prof. Hamid Arabnia and his puppet?s activities at > http://worldcomp-fake-bogus.blogspot.com Search Google using the > keyword worldcomp fake for > additional links. > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard at arbee-design.co.uk Wed May 1 10:23:34 2013 From: richard at arbee-design.co.uk (Richard Barran) Date: Wed, 1 May 2013 09:23:34 +0100 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: Spam? Sounds on-topic for a Python mailing list :-) On 1 May 2013, at 00:52, Stestagg wrote: > Is this just spam? It certainly has that feel about it.. > > > On Wed, May 1, 2013 at 12:12 AM, wrote: > Biggest Fake Conference in Computer Science > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk -------------- next part -------------- An HTML attachment was scrubbed... URL: From morton.thomas at googlemail.com Wed May 1 10:32:57 2013 From: morton.thomas at googlemail.com (Thomas Morton) Date: Wed, 1 May 2013 09:32:57 +0100 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: They've been sending it to loads of list; Wikimedia lists got bazillions of copies :D Tom On 1 May 2013 09:23, Richard Barran wrote: > Spam? Sounds on-topic for a Python mailing list :-) > > On 1 May 2013, at 00:52, Stestagg wrote: > > Is this just spam? It certainly has that feel about it.. > > > On Wed, May 1, 2013 at 12:12 AM, wrote: > >> Biggest Fake Conference in Computer Science >> >> >> >> _______________________________________________ >> python-uk mailing list >> python-uk at python.org >> http://mail.python.org/mailman/listinfo/python-uk >> > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stestagg at gmail.com Wed May 1 11:39:17 2013 From: stestagg at gmail.com (Stestagg) Date: Wed, 1 May 2013 10:39:17 +0100 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: Sorry, I was being prejudiced. The general tone of the message, and the grammatical mistakes made me think this might have been an attempt to sell me performance enhancing drugs or something. I guess it seems genuine, and interesting take on spreading the message. Steve On Wed, May 1, 2013 at 9:32 AM, Thomas Morton wrote: > They've been sending it to loads of list; Wikimedia lists got bazillions > of copies :D > > Tom > > > On 1 May 2013 09:23, Richard Barran wrote: > >> Spam? Sounds on-topic for a Python mailing list :-) >> >> On 1 May 2013, at 00:52, Stestagg wrote: >> >> Is this just spam? It certainly has that feel about it.. >> >> >> On Wed, May 1, 2013 at 12:12 AM, wrote: >> >>> Biggest Fake Conference in Computer Science >>> >>> >>> >>> _______________________________________________ >>> python-uk mailing list >>> python-uk at python.org >>> http://mail.python.org/mailman/listinfo/python-uk >>> >> >> _______________________________________________ >> python-uk mailing list >> python-uk at python.org >> http://mail.python.org/mailman/listinfo/python-uk >> >> >> >> _______________________________________________ >> python-uk mailing list >> python-uk at python.org >> http://mail.python.org/mailman/listinfo/python-uk >> >> > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dougal85 at gmail.com Wed May 1 11:45:29 2013 From: dougal85 at gmail.com (Dougal Matthews) Date: Wed, 1 May 2013 10:45:29 +0100 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: <0ECFA23E34DE4C4A8E7B8DE1E3D10D00@gmail.com> On Wednesday, 1 May 2013 at 10:39, Stestagg wrote: > Sorry, I was being prejudiced. > > The general tone of the message, and the grammatical mistakes made me think this might have been an attempt to sell me performance enhancing drugs or something. > > I guess it seems genuine, and interesting take on spreading the message. FWIW. I consider this spam. If for no other reason I have received it before on Python lists and today I got 6 copies due to being on multiple lists. The sender also uses a throwaway anonymous e-mail address. Dougal -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim at red56.co.uk Wed May 1 11:57:14 2013 From: tim at red56.co.uk (Tim Diggins) Date: Wed, 1 May 2013 10:57:14 +0100 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: Interesting. Agree it is roughly on-topic, It's doesn't seem to be a rogue or fraudulent email. But also it does sound like it's unsolicited bulk mail. I'd rather leave it up to a Bayesian filter to decide if that all means it's spam or not. ;-) On 1 May 2013 09:32, Thomas Morton wrote: > They've been sending it to loads of list; Wikimedia lists got bazillions > of copies :D > > Tom > > > On 1 May 2013 09:23, Richard Barran wrote: > >> Spam? Sounds on-topic for a Python mailing list :-) >> >> On 1 May 2013, at 00:52, Stestagg wrote: >> >> Is this just spam? It certainly has that feel about it.. >> >> >> On Wed, May 1, 2013 at 12:12 AM, wrote: >> >>> Biggest Fake Conference in Computer Science >>> >>> >>> >>> _______________________________________________ >>> python-uk mailing list >>> python-uk at python.org >>> http://mail.python.org/mailman/listinfo/python-uk >>> >> >> _______________________________________________ >> python-uk mailing list >> python-uk at python.org >> http://mail.python.org/mailman/listinfo/python-uk >> >> >> >> _______________________________________________ >> python-uk mailing list >> python-uk at python.org >> http://mail.python.org/mailman/listinfo/python-uk >> >> > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From J.Gould at austinfraser.com Wed May 1 12:02:53 2013 From: J.Gould at austinfraser.com (Jon Gould) Date: Wed, 1 May 2013 10:02:53 +0000 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: Have you got his number? Regards Jon Gould | Team Leader Austin Fraser Limited | Digital Contracts Tel: 0118 952 0153 | Skype: JonWebJobs | Twitter: @JonWebJobs [cid:image001.jpg at 01CDEA85.49E692D0] [https://si0.twimg.com/a/1357250038/images/resources/twitter-bird-white-on-blue.png] [http://t1.gstatic.com/images?q=tbn:ANd9GcRa2LDcp4VF4rQJSioaQWwdsJjsHnaJUYwq6gHNsdnZJWlngPapjQ] [http://t1.gstatic.com/images?q=tbn:ANd9GcSVG9csvWvSkdYSn0P9veokPAM7Qc-NQyzwVubIqvm8hopWs1Ru] From: python-uk [mailto:python-uk-bounces+j.gould=austinfraser.com at python.org] On Behalf Of Tim Diggins Sent: 01 May 2013 10:57 To: UK Python Users Subject: Re: [python-uk] Biggest Fake Conference in Computer Science Interesting. Agree it is roughly on-topic, It's doesn't seem to be a rogue or fraudulent email. But also it does sound like it's unsolicited bulk mail. I'd rather leave it up to a Bayesian filter to decide if that all means it's spam or not. ;-) On 1 May 2013 09:32, Thomas Morton > wrote: They've been sending it to loads of list; Wikimedia lists got bazillions of copies :D Tom On 1 May 2013 09:23, Richard Barran > wrote: Spam? Sounds on-topic for a Python mailing list :-) On 1 May 2013, at 00:52, Stestagg > wrote: Is this just spam? It certainly has that feel about it.. On Wed, May 1, 2013 at 12:12 AM, > wrote: Biggest Fake Conference in Computer Science _______________________________________________ python-uk mailing list python-uk at python.org http://mail.python.org/mailman/listinfo/python-uk _______________________________________________ python-uk mailing list python-uk at python.org http://mail.python.org/mailman/listinfo/python-uk _______________________________________________ python-uk mailing list python-uk at python.org http://mail.python.org/mailman/listinfo/python-uk _______________________________________________ python-uk mailing list python-uk at python.org http://mail.python.org/mailman/listinfo/python-uk ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 1322 bytes Desc: image001.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 946 bytes Desc: image002.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.jpg Type: image/jpeg Size: 1138 bytes Desc: image003.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.jpg Type: image/jpeg Size: 1061 bytes Desc: image004.jpg URL: From J.Gould at austinfraser.com Wed May 1 12:10:51 2013 From: J.Gould at austinfraser.com (Jon Gould) Date: Wed, 1 May 2013 10:10:51 +0000 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: Sorry about my fat fingers! Ignore the last email, I was replying to someone else. Regards Jon Gould | Team Leader Austin Fraser Limited | Digital Contracts Tel: 0118 952 0153 | Skype: JonWebJobs | Twitter: @JonWebJobs [cid:image001.jpg at 01CDEA85.49E692D0] [https://si0.twimg.com/a/1357250038/images/resources/twitter-bird-white-on-blue.png] [http://t1.gstatic.com/images?q=tbn:ANd9GcRa2LDcp4VF4rQJSioaQWwdsJjsHnaJUYwq6gHNsdnZJWlngPapjQ] [http://t1.gstatic.com/images?q=tbn:ANd9GcSVG9csvWvSkdYSn0P9veokPAM7Qc-NQyzwVubIqvm8hopWs1Ru] From: Jon Gould Sent: 01 May 2013 11:03 To: 'UK Python Users' Subject: RE: [python-uk] Biggest Fake Conference in Computer Science Have you got his number? Regards Jon Gould | Team Leader Austin Fraser Limited | Digital Contracts Tel: 0118 952 0153 | Skype: JonWebJobs | Twitter: @JonWebJobs [cid:image001.jpg at 01CDEA85.49E692D0] [https://si0.twimg.com/a/1357250038/images/resources/twitter-bird-white-on-blue.png] [http://t1.gstatic.com/images?q=tbn:ANd9GcRa2LDcp4VF4rQJSioaQWwdsJjsHnaJUYwq6gHNsdnZJWlngPapjQ] [http://t1.gstatic.com/images?q=tbn:ANd9GcSVG9csvWvSkdYSn0P9veokPAM7Qc-NQyzwVubIqvm8hopWs1Ru] From: python-uk [mailto:python-uk-bounces+j.gould=austinfraser.com at python.org] On Behalf Of Tim Diggins Sent: 01 May 2013 10:57 To: UK Python Users Subject: Re: [python-uk] Biggest Fake Conference in Computer Science Interesting. Agree it is roughly on-topic, It's doesn't seem to be a rogue or fraudulent email. But also it does sound like it's unsolicited bulk mail. I'd rather leave it up to a Bayesian filter to decide if that all means it's spam or not. ;-) On 1 May 2013 09:32, Thomas Morton > wrote: They've been sending it to loads of list; Wikimedia lists got bazillions of copies :D Tom On 1 May 2013 09:23, Richard Barran > wrote: Spam? Sounds on-topic for a Python mailing list :-) On 1 May 2013, at 00:52, Stestagg > wrote: Is this just spam? It certainly has that feel about it.. On Wed, May 1, 2013 at 12:12 AM, > wrote: Biggest Fake Conference in Computer Science _______________________________________________ python-uk mailing list python-uk at python.org http://mail.python.org/mailman/listinfo/python-uk _______________________________________________ python-uk mailing list python-uk at python.org http://mail.python.org/mailman/listinfo/python-uk _______________________________________________ python-uk mailing list python-uk at python.org http://mail.python.org/mailman/listinfo/python-uk _______________________________________________ python-uk mailing list python-uk at python.org http://mail.python.org/mailman/listinfo/python-uk ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 1322 bytes Desc: image001.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 946 bytes Desc: image002.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.jpg Type: image/jpeg Size: 1138 bytes Desc: image003.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.jpg Type: image/jpeg Size: 1061 bytes Desc: image004.jpg URL: From michael at brunton-spall.co.uk Wed May 1 14:59:27 2013 From: michael at brunton-spall.co.uk (Michael Brunton-Spall) Date: Wed, 1 May 2013 13:59:27 +0100 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: <0ECFA23E34DE4C4A8E7B8DE1E3D10D00@gmail.com> References: <20130430231238.7B5D7E6736@smtp.hushmail.com> <0ECFA23E34DE4C4A8E7B8DE1E3D10D00@gmail.com> Message-ID: <20130501125927.GB27486@gnm31330.int.gnl> On Wed, May 01, 2013 at 10:45:29AM +0100, Dougal Matthews wrote: > On Wednesday, 1 May 2013 at 10:39, Stestagg wrote: > > Sorry, I was being prejudiced. > > > > The general tone of the message, and the grammatical mistakes made me think this might have been an attempt to sell me performance enhancing drugs or something. > > > > I guess it seems genuine, and interesting take on spreading the message. > > FWIW. I consider this spam. If for no other reason I have received it before > on Python lists and today I got 6 copies due to being on multiple lists. The > sender also uses a throwaway anonymous e-mail address. > > Dougal > I agree, I felt it was spam, not only because it was unsolicted mail, but also because it makes difficult to verify accusations from an anonymous account without any reference to sources. MBS > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk From general.mooney at gmail.com Wed May 1 10:33:49 2013 From: general.mooney at gmail.com (=?UTF-8?Q?Ciar=C3=A1n_Mooney?=) Date: Wed, 1 May 2013 09:33:49 +0100 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: Seeing as this comes from a hushmail.com account (anonymous service I believe), and says some not very nice things about a particular individual I think it can be taken with a pinch of salt. Academic conferences don't make mega-bucks, even if you ran a corrupt one you'd have to offer something else along with it (visas for entry). Which would get you noticed by the authorities pretty quickly. If you are going to make these kind of claims against someone you either need to have a good reason to hide your name or you publish and be damned. Mr Arabnia's side of the story seems a lot more plausible. http://www.redandblack.com/news/professor-deals-with-elaborate-cyber-attack/article_03344d54-cd7f-5753-a870-e3bcee5c87ce.html On 1 May 2013 09:23, Richard Barran wrote: > Spam? Sounds on-topic for a Python mailing list :-) > > On 1 May 2013, at 00:52, Stestagg wrote: > > Is this just spam? It certainly has that feel about it.. > > > On Wed, May 1, 2013 at 12:12 AM, wrote: >> >> Biggest Fake Conference in Computer Science >> >> >> >> _______________________________________________ >> python-uk mailing list >> python-uk at python.org >> http://mail.python.org/mailman/listinfo/python-uk > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > From matthew at unsolvable.org Wed May 1 11:52:33 2013 From: matthew at unsolvable.org (Matthew Webber) Date: Wed, 1 May 2013 10:52:33 +0100 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: It is irrelevant whether it is genuine or not, the message is completely off-topic for this forum, and should not have been posted here. It looks like someone has got really worked up about something (rightly or wrongly), and decided to tell the entire work about it (definitely wrong). That's my take, anyhow. Matthew On 1 May 2013 10:39, Stestagg wrote: > Sorry, I was being prejudiced. > > The general tone of the message, and the grammatical mistakes made me > think this might have been an attempt to sell me performance enhancing > drugs or something. > > I guess it seems genuine, and interesting take on spreading the message. > > Steve > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at python.org Wed May 1 18:17:23 2013 From: chris at python.org (Chris Withers) Date: Wed, 01 May 2013 17:17:23 +0100 Subject: [python-uk] Biggest Fake Conference in Computer Science In-Reply-To: References: <20130430231238.7B5D7E6736@smtp.hushmail.com> Message-ID: <51814013.6010800@python.org> A fat fingered recruiter? Surely not! On 01/05/2013 11:10, Jon Gould wrote: > Sorry about my fat fingers! Ignore the last email, I was replying to > someone else. > > Regards > > *Jon Gould *| Team Leader > > *Austin Fraser Limited *| Digital Contracts > > *Tel:*0118 952 0153**| *Skype:* JonWebJobs | *Twitter: *@JonWebJobs > > cid:image001.jpg at 01CDEA85.49E692D0 > https://si0.twimg.com/a/1357250038/images/resources/twitter-bird-white-on-blue.png > http://t1.gstatic.com/images?q=tbn:ANd9GcRa2LDcp4VF4rQJSioaQWwdsJjsHnaJUYwq6gHNsdnZJWlngPapjQ > http://t1.gstatic.com/images?q=tbn:ANd9GcSVG9csvWvSkdYSn0P9veokPAM7Qc-NQyzwVubIqvm8hopWs1Ru > > *From:*Jon Gould > *Sent:* 01 May 2013 11:03 > *To:* 'UK Python Users' > *Subject:* RE: [python-uk] Biggest Fake Conference in Computer Science > > Have you got his number? > > Regards > > *Jon Gould *| Team Leader > > *Austin Fraser Limited *| Digital Contracts > > *Tel:*0118 952 0153**| *Skype:* JonWebJobs | *Twitter: *@JonWebJobs > > cid:image001.jpg at 01CDEA85.49E692D0 > https://si0.twimg.com/a/1357250038/images/resources/twitter-bird-white-on-blue.png > http://t1.gstatic.com/images?q=tbn:ANd9GcRa2LDcp4VF4rQJSioaQWwdsJjsHnaJUYwq6gHNsdnZJWlngPapjQ > http://t1.gstatic.com/images?q=tbn:ANd9GcSVG9csvWvSkdYSn0P9veokPAM7Qc-NQyzwVubIqvm8hopWs1Ru > > *From:*python-uk > [mailto:python-uk-bounces+j.gould=austinfraser.com at python.org > ] *On > Behalf Of *Tim Diggins > *Sent:* 01 May 2013 10:57 > *To:* UK Python Users > *Subject:* Re: [python-uk] Biggest Fake Conference in Computer Science > > Interesting. > > Agree it is roughly on-topic, It's doesn't seem to be a rogue or > fraudulent email. > > But also it does sound like it's unsolicited bulk mail. > > I'd rather leave it up to a Bayesian filter to decide if that all means > it's spam or not. ;-) > > On 1 May 2013 09:32, Thomas Morton > wrote: > > They've been sending it to loads of list; Wikimedia lists got bazillions > of copies :D > > Tom > > On 1 May 2013 09:23, Richard Barran > wrote: > > Spam? Sounds on-topic for a Python mailing list :-) > > On 1 May 2013, at 00:52, Stestagg > wrote: > > Is this just spam? It certainly has that feel about it.. > > On Wed, May 1, 2013 at 12:12 AM, > wrote: > > Biggest Fake Conference in Computer Science > > > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > > ______________________________________________________________________ > This email has been scanned by the Symantec Email Security.cloud service. > For more information please visit http://www.symanteccloud.com > ______________________________________________________________________ > > > ______________________________________________________________________ > This email has been scanned by the Symantec Email Security.cloud service. > For more information please visit http://www.symanteccloud.com > ______________________________________________________________________ > > ______________________________________________________________________ > This email has been scanned by the Symantec Email Security.cloud service. > For more information please visit http://www.symanteccloud.com > ______________________________________________________________________ > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From andy at reportlab.com Thu May 2 12:15:01 2013 From: andy at reportlab.com (Andy Robinson) Date: Thu, 2 May 2013 11:15:01 +0100 Subject: [python-uk] Posting is for members - please remember which email you use! Message-ID: I have approved quite a lot of emails today and yesterday from well-known UK Pythonauts, which were blocked for "posting by non-members to members-only lists". Luckily, I have been around and not manic. If you have multiple email addresses, can you try to either remember which one is registered with the list and use that, or sign up again with your new one? Thanks, -- Andy Robinson From fuzzyman at voidspace.org.uk Thu May 2 12:17:08 2013 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 2 May 2013 11:17:08 +0100 Subject: [python-uk] Posting is for members - please remember which email you use! In-Reply-To: References: Message-ID: On 2 May 2013, at 11:15, Andy Robinson wrote: > I have approved quite a lot of emails today and yesterday from > well-known UK Pythonauts, which were blocked for "posting by > non-members to members-only lists". Luckily, I have been around and > not manic. If you have multiple email addresses, can you try to > either remember which one is registered with the list and use that, or > sign up again with your new one? > If you have multiple addresses you can sign up with them and then turn off email delivery. That allows you to post from any of them, but they only get delivered once. Michael > Thanks, > > -- > Andy Robinson > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html From rachid.belaid at gmail.com Fri May 3 11:37:27 2013 From: rachid.belaid at gmail.com (Rachid Belaid) Date: Fri, 3 May 2013 10:37:27 +0100 Subject: [python-uk] First Pyramid Meetup in London 7th May Message-ID: Hi, Last time we discussed about doing a Pyramid meetup, peoples were keen so now it's happening. You will find the details about this first meetup here http://www.meetup.com/The-London-Pyramid-Group/events/114457692/ We have 2 talks planned but there is still one slot available for anybody who want to share any Pyramid / Pylons / SQLAlchemy experiences. If somebody is keen about doing the third talk, drop me an email before Monday. Thanks Rach PS: the meetup will be followed by a visit to the pub obviously if you cannot make it for the talks. -- Rach Belaid Twitter: @rachbelaid -------------- next part -------------- An HTML attachment was scrubbed... URL: From general.mooney at gmail.com Sat May 4 12:13:48 2013 From: general.mooney at gmail.com (=?UTF-8?Q?Ciar=C3=A1n_Mooney?=) Date: Sat, 4 May 2013 11:13:48 +0100 Subject: [python-uk] Maze Generator - Team 3 Message-ID: To the members of team three and other dojo'ers, Below is a link to my maze generator, using David's idea of procedural generator (rather than the algorithm Dan described). https://github.com/ciaranmooney/procMaze I think it works, and only took about 20 mins to write in the end. The mazes it makes are not the best in the world. I've no idea what we were doing wrong, but if someone from Team 3 (David, Caroline, or Tony) could send me the code we wrote I'd love to have another look at it. Cheers, Ciar?n From lord.mauve at gmail.com Sat May 4 13:00:08 2013 From: lord.mauve at gmail.com (Daniel Pope) Date: Sat, 4 May 2013 12:00:08 +0100 Subject: [python-uk] Maze Generator - Team 3 In-Reply-To: References: Message-ID: My 3D maze-drawing code is at https://bitbucket.org/lordmauve/ldnpydojo-s4e9 Integrating a maze just requires implementing itercells(). On 4 May 2013 11:13, Ciar?n Mooney wrote: > To the members of team three and other dojo'ers, > > Below is a link to my maze generator, using David's idea of procedural > generator (rather than the algorithm Dan described). > > https://github.com/ciaranmooney/procMaze > > I think it works, and only took about 20 mins to write in the end. The > mazes it makes are not the best in the world. > > I've no idea what we were doing wrong, but if someone from Team 3 > (David, Caroline, or Tony) could send me the code we wrote I'd love to > have another look at it. > > Cheers, > > Ciar?n > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From whykay at python.ie Mon May 6 01:11:50 2013 From: whykay at python.ie (Vicky Lee - Python Ireland) Date: Mon, 6 May 2013 00:11:50 +0100 Subject: [python-uk] [ANN] PyCon Ireland 2013 Early Bird tickets and CFP Message-ID: Hi All, WHEN & WHERE ============== Conference: 12th - 13th October, Burlington Hotel, Dublin Sprints: 14th - 15th October, College of Computer Training (CCT), Dublin TICKETS ======== *Early Bird: ?50-55* Corporate tickets: ?200 Student: ?30 Standard: ?60-65 (Purchase tickets via http://python.ie/pycon/2013) CFP ==== Submit talks to http://bit.ly/speakerpyconie CALL FOR SPONSORS =================== Details about sponsorship: http://bit.ly/sponsorpyconie If you have any questions, please email pycon at python.ie. Cheers, /// Vicky (PyCon Ireland co-Chair) Python Ireland co-Chair / Treasurer EuroPython Board PSF member -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike at mikedeplume.com Mon May 6 16:43:47 2013 From: mike at mikedeplume.com (MikedePlume) Date: Mon, 06 May 2013 15:43:47 +0100 Subject: [python-uk] London Dojo mazement - team1 Message-ID: <1367851427.5658.8.camel@oberto> Just to show that it _can_ be done I have updated https://bitbucket.org/mikedeplume/physical-turtle with example_random.py which uses Physical Turtle to generate a random walk that does not cross itself. Probably not something to have attempted without some prior experience, and many thanks to Team 1 for providing that experience :-) Mike S. From tehunger at gmail.com Mon May 6 20:26:04 2013 From: tehunger at gmail.com (Thomas Hunger) Date: Mon, 6 May 2013 19:26:04 +0100 Subject: [python-uk] more maze Message-ID: Here's an implementation of a random spanning tree where the nodes are coordinates on a plane so one can render them later: https://gist.github.com/teh/5526976 ~ From mail at timgolden.me.uk Mon May 6 20:33:52 2013 From: mail at timgolden.me.uk (Tim Golden) Date: Mon, 06 May 2013 19:33:52 +0100 Subject: [python-uk] more maze In-Reply-To: References: Message-ID: <5187F790.7030402@timgolden.me.uk> On 06/05/2013 19:26, Thomas Hunger wrote: > Here's an implementation of a random spanning tree where the nodes are > coordinates on a plane so one can render them later: > > https://gist.github.com/teh/5526976 Thanks, Thomas. For those wondering, last Thursday's London Python Dojo was all about generating mazes. Most teams didn't have time to fulfil their solution's entire potential and evidently a few people went home buzzing with ideas or alternatives. Hence the flow of messages to the list promoting maze solutions. Just so you knew... The next Dojo should be on the first Thursday of June, venue to be announced. Keep an eye on this list and/or follow @ldnpydojo. TJG From lord.mauve at gmail.com Tue May 7 22:41:00 2013 From: lord.mauve at gmail.com (Daniel Pope) Date: Tue, 7 May 2013 21:41:00 +0100 Subject: [python-uk] Maze Generator - Team 3 In-Reply-To: References: Message-ID: On 4 May 2013 11:13, Ciar?n Mooney wrote: > Below is a link to my maze generator, using David's idea of procedural > generator (rather than the algorithm Dan described). > Seeing as everyone seems to still be hacking on this, I took a stab at implementing the algorithm I was suggesting in the dojo, and I hooked it up to the 3D maze renderer: https://bitbucket.org/lordmauve/ldnpydojo-s4e9 -------------- next part -------------- An HTML attachment was scrubbed... URL: From salim.fadhley at baml.com Thu May 9 14:29:31 2013 From: salim.fadhley at baml.com (Fadhley, Salim) Date: Thu, 09 May 2013 12:29:31 +0000 Subject: [python-uk] Bank of America (Canary Wharf) - we are still hiring! Message-ID: My team is still on the look-out for Python people. The job-spec below is the same one I posted a month ago. We are looking for people with software engineering or computer-science background. We actually have a number of roles across various teams - some of which might be more attractive to developers from a math/algo background. Feel free to call me if you have any questions, all the details are at the end. Bank of America, Reconciliations Team (Canary Wharf, London, UK) ================================================================ Job Description: ---------------- We are currently recruiting Python developers to join the Quartz Reconciliations team. Quartz is a company-wide project to consolidate business systems into a single Python-based platform. Quartz provides grid-computing, storage, a consistent developer environment, and financial/UI APIs. The Reconciliations team is responsible for building tools which help teams within the organization improve the quality and consistency of data across a large number of financial systems. We are looking for candidates with the aptitude to grapple with the complexity of the ever-growing Quartz platform. We are looking for experienced Python developers who can act as Python mentors to other developers and provide leadership in design, development, testing and delivery of Python code. Successful candidates will play a role in the full development cycle of our products, working within an Agile (Scrum) development process. Our primary development language is Python, however skills in other technologies such as relational and document-oriented databases may be beneficial. We are interested in meeting candidates from all backgrounds (not just finance), with a broad technological knowledge. We value knowledge in aspects of computing and technology which do not have a direct application on our project. Requirements ------------ * Previous experience as a senior Python developer * Fluency in the Python language + Standard libraries * Experience in test-driven development Contact Info: ------------- * Contact: Salim Fadhley / Stephen Brown * E-mail contact: salim.fadhley at baml.com / stephen.d.brown at baml.com * Other Contact Info: t. 0207 9951134 / 0207 9961845 * No telecommuting * No headhunters! ---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. From salim.fadhley at baml.com Thu May 9 19:43:36 2013 From: salim.fadhley at baml.com (Fadhley, Salim) Date: Thu, 09 May 2013 17:43:36 +0000 Subject: [python-uk] British Library hack event May 28-29 Message-ID: I've been invited (and will probably attend) a hack event at the British Library - the goal is to build help the BL find ways to archive community radio. I've been personally involved with Resonance FM since 2003. In case you've never heard of it, Resonance is probably the world's most bizarre radio station. It provides a home for audio artists and musicians who might never get exposure from commercial radio. Resonance has shared an archive of more than a decade's worth of audio on a bunch of hard-disk drives. The challenge of this event is to find a way to make sense of it all. The eventual goal will be to allow the British Library to host Resonance FM's audio in much the same way that it currently does for the BBC World Service. If you are interested, the registration URL is: http://labs.bl.uk/Competition+2013+-+Hack+Event Sal ---------- Forwarded message ---------- From: "Wilson, Paul" Date: 9 May 2013 17:36 Subject: British Library hack event May 28-29 To: Cc: Dear Salim, ? I?m curator of radio here at the British Library, which is now home to the Resonance FM Archive. Chris Weaver has suggested that you may be interested in a two-day hack event which BL Labs are hosting at the Library?s Euston Road premises on Tues 28th and Wed 29th May. The aim of this event is to resolve some technical issues which are currently preventing us from (a) archiving the Resonance FM collection into the Library?s Digital Library System (which will guarantee its survival alongside all the Library?s other digital media for, in theory, forever!) and (b) cataloguing the collection to the Library?s online catalogues where researchers and other interested users can discover the collection?s contents. If these problems are resolved at the event we also then hope to start building, with Resonance?s approval/assistance, a (legally compliant) online collection of programmes from the archive which can then be used by the wider community for creative or educational purposes, or just for the pleasure of listening. ? The collection which we have taken into the Library is all of the extant audio since 2002 which we currently hold on a set of hard drives, along with the schedule data (from 2002 to about July 2007 as Filemaker Pro and from then onwards as i-Cal). The recordings we?ll be focussing on at the event are the original ?day files? ? Resonance?s recordings of transmission, typically between one and four files per day. What we are hoping to accomplish at the hack event is to use the transmission dates and start times inherent in the file names, along with the schedule data, to segment the day files into individual programmes and link both in a spreadsheet which we can then use to achieve our stated objectives. ? If this is accomplished quickly at the event we may also wish to look at resolving some other issues relating to preparation of the Resonance collection, but the hack event will also be presenting some other British Library collections which may be of interest to you. ? If you are able to join us for this event ? or just part of it even ? then you?d be very welcome. You can find out more and register here: ? http://labs.bl.uk/Competition+2013+-+Hack+Event ? and if you have any questions before the day please feel free to ask. ? Best wishes ? ? Paul ? _______________________________________________________ ? Paul Wilson Curator, Radio The British Library 96 Euston Road London NW1 2DB Tel: 020 7412 7446 Email: pwilson at bl.uk Web: www.bl.uk/soundarchive ? ************************************************************************** Experience the British Library online at www.bl.uk ? The British Library?s latest Annual Report and Accounts : www.bl.uk/aboutus/annrep/index.html ? Help the British Library conserve the world's knowledge. Adopt a Book. www.bl.uk/adoptabook ? The Library's St Pancras site is WiFi - enabled ? ************************************************************************* ? The information contained in this e-mail is confidential and may be legally privileged. It is intended for the addressee(s) only. If you are not the intended recipient, please delete this e-mail and notify the postmaster at bl.uk : The contents of this e-mail must not be disclosed or copied without the sender's consent. ? The statements and opinions expressed in this message are those of the author and do not necessarily reflect those of the British Library. The British Library does not take any responsibility for the views of the author. ? ************************************************************************* ?Think before you print ---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. From ed.hartley at gmail.com Thu May 9 20:27:23 2013 From: ed.hartley at gmail.com (E Hartley) Date: Thu, 9 May 2013 19:27:23 +0100 Subject: [python-uk] British Library hack event May 28-29 In-Reply-To: References: Message-ID: Hi, I'd very much like to attend this as I've a long standing interest in media archiving and meta-data from my involvement in MPEG and standardisation. However circumstances may preclude me attending, so in any case I've forwarded the message to the BSI reflector concerned with MPEG and media metadata as I know the workshop will be of interest to other members of that list. Best Regards Ed Hartley On 9 May 2013, at 18:43, "Fadhley, Salim" wrote: > I've been invited (and will probably attend) a hack event at the British Library - the goal is to build help the BL find ways to archive community radio. > > I've been personally involved with Resonance FM since 2003. In case you've never heard of it, Resonance is probably the world's most bizarre radio station. It provides a home for audio artists and musicians who might never get exposure from commercial radio. > > Resonance has shared an archive of more than a decade's worth of audio on a bunch of hard-disk drives. The challenge of this event is to find a way to make sense of it all. The eventual goal will be to allow the British Library to host Resonance FM's audio in much the same way that it currently does for the BBC World Service. > > If you are interested, the registration URL is: > http://labs.bl.uk/Competition+2013+-+Hack+Event > > Sal > > ---------- Forwarded message ---------- > From: "Wilson, Paul" > Date: 9 May 2013 17:36 > Subject: British Library hack event May 28-29 > To: > Cc: > > Dear Salim, > > I?m curator of radio here at the British Library, which is now home to the Resonance FM Archive. Chris Weaver has suggested that you may be interested in a two-day hack event which BL Labs are hosting at the Library?s Euston Road premises on Tues 28th and Wed 29th May. The aim of this event is to resolve some technical issues which are currently preventing us from (a) archiving the Resonance FM collection into the Library?s Digital Library System (which will guarantee its survival alongside all the Library?s other digital media for, in theory, forever!) and (b) cataloguing the collection to the Library?s online catalogues where researchers and other interested users can discover the collection?s contents. If these problems are resolved at the event we also then hope to start building, with Resonance?s approval/assistance, a (legally compliant) online collection of programmes from the archive which can then be used by the wider community for creative or educational purposes, or just for the pleasure of listening. > > The collection which we have taken into the Library is all of the extant audio since 2002 which we currently hold on a set of hard drives, along with the schedule data (from 2002 to about July 2007 as Filemaker Pro and from then onwards as i-Cal). The recordings we?ll be focussing on at the event are the original ?day files? ? Resonance?s recordings of transmission, typically between one and four files per day. What we are hoping to accomplish at the hack event is to use the transmission dates and start times inherent in the file names, along with the schedule data, to segment the day files into individual programmes and link both in a spreadsheet which we can then use to achieve our stated objectives. > > If this is accomplished quickly at the event we may also wish to look at resolving some other issues relating to preparation of the Resonance collection, but the hack event will also be presenting some other British Library collections which may be of interest to you. > > If you are able to join us for this event ? or just part of it even ? then you?d be very welcome. You can find out more and register here: > > http://labs.bl.uk/Competition+2013+-+Hack+Event > > and if you have any questions before the day please feel free to ask. > > Best wishes > > > Paul > > _______________________________________________________ > > Paul Wilson > Curator, Radio > The British Library > 96 Euston Road > London NW1 2DB > Tel: 020 7412 7446 > Email: pwilson at bl.uk > Web: www.bl.uk/soundarchive > > ************************************************************************** > Experience the British Library online at www.bl.uk > > The British Library?s latest Annual Report and Accounts : www.bl.uk/aboutus/annrep/index.html > > Help the British Library conserve the world's knowledge. Adopt a Book. www.bl.uk/adoptabook > > The Library's St Pancras site is WiFi - enabled > > ************************************************************************* > > The information contained in this e-mail is confidential and may be legally privileged. It is intended for the addressee(s) only. If you are not the intended recipient, please delete this e-mail and notify the postmaster at bl.uk : The contents of this e-mail must not be disclosed or copied without the sender's consent. > > The statements and opinions expressed in this message are those of the author and do not necessarily reflect those of the British Library. The British Library does not take any responsibility for the views of the author. > > ************************************************************************* > Think before you print > > ---------------------------------------------------------------------- > This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk From tom at viner.tv Sun May 12 11:01:21 2013 From: tom at viner.tv (Tom Viner) Date: Sun, 12 May 2013 10:01:21 +0100 Subject: [python-uk] Fwd: more maze In-Reply-To: <5187F790.7030402@timgolden.me.uk> References: <5187F790.7030402@timgolden.me.uk> Message-ID: I mentioned some comments to Thomas about his elegant random spanning tree maze script and he insisted that I post here so everyone can see the correction and extensions . First there was a minor bug where the initial location wasn't included in the seen set, that meant you could sometimes get cycles in the supposedly acyclic graph. [image: Inline images 1](Note loop at bottom left) In testing this I found I needed a way repeating a certain random maze, so I added this to the start: seed = sys.argv[1] if sys.argv[1:2] else random.randint(0, 999) print "Replay with python maze.py", seed random.seed(int(seed)) which picks a random seed unless you pass one in on the command line. Then I wanted to see how the backtracking worked, so I tried slowing down the turtle but there was still a bit of a pause before anything was drawn. This was because the maze was being pre-computed, then drawn afterwards. Turning the maze algorithm into a generator fixed this. Finally I asked Thomas how he'd test if any given maze was solvable. He said "The spanning tree connects all nodes. There are no unconnected subgraphs. For that reason you can pick any two points as start and end and have a solvable maze". At this point I realised I'd been looking at the maze drawing backwards: he'd intended the lines the turtle draws to be the *paths*, and *everything else* was walls. Rather than calculating exactly where all the walls would be, to contain a given path, I just added a background and made the turtle carve out the path of the maze. I also added entrances when the path hits the edge. So now we have a familiar looking maze: [image: Inline images 1] Instructions: Pick 2 opposite entrances and try to get from one to the other. I wonder if we could let Mike's Physical Turtle loose on this... Tom ---------- Forwarded message ---------- From: Tim Golden Date: 6 May 2013 19:33 Subject: Re: [python-uk] more maze To: python-uk at python.org On 06/05/2013 19:26, Thomas Hunger wrote: > Here's an implementation of a random spanning tree where the nodes are > coordinates on a plane so one can render them later: > > https://gist.github.com/teh/**5526976 > Thanks, Thomas. For those wondering, last Thursday's London Python Dojo was all about generating mazes. Most teams didn't have time to fulfil their solution's entire potential and evidently a few people went home buzzing with ideas or alternatives. Hence the flow of messages to the list promoting maze solutions. Just so you knew... The next Dojo should be on the first Thursday of June, venue to be announced. Keep an eye on this list and/or follow @ldnpydojo. TJG ______________________________**_________________ python-uk mailing list python-uk at python.org http://mail.python.org/**mailman/listinfo/python-uk -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot from 2013-05-07 23:55:52.png Type: image/png Size: 706 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot from 2013-05-12 01:46:09.png Type: image/png Size: 1060 bytes Desc: not available URL: From tehunger at gmail.com Sun May 12 12:54:20 2013 From: tehunger at gmail.com (Thomas Hunger) Date: Sun, 12 May 2013 11:54:20 +0100 Subject: [python-uk] more maze In-Reply-To: References: <5187F790.7030402@timgolden.me.uk> Message-ID: Very cool. Thanks for the nice write-up and the fixes! Daniel Pope's maze generator, mentioned here [1], is a different implementation of the same algorithm. ~ [1] http://mail.python.org/pipermail/python-uk/2013-May/002942.html On 12 May 2013 10:01, Tom Viner wrote: > I mentioned some comments to Thomas about his elegant random spanning > tree maze script and he insisted > that I post here so everyone can see the correction and extensions > . > > First there was a minor bug where the initial location wasn't included in > the seen set, that meant you could sometimes get cycles in > the supposedly acyclic graph. > > [image: Inline images 1](Note loop at bottom left) > > In testing this I found I needed a way repeating a certain random maze, so > I added this to the start: > > seed = sys.argv[1] if sys.argv[1:2] else random.randint(0, 999) > print "Replay with python maze.py", seed > random.seed(int(seed)) > > which picks a random seed unless you pass one in on the command line. > > Then I wanted to see how the backtracking worked, so I tried slowing down > the turtle but there was still a bit of a pause before anything was drawn. > This was because the maze was being pre-computed, then drawn afterwards. > Turning the maze algorithm into a generator fixed this. > > Finally I asked Thomas how he'd test if any given maze was solvable. He > said "The spanning tree connects all nodes. There are no unconnected > subgraphs. For that reason you can pick any two points as start and end and > have a solvable maze". > > At this point I realised I'd been looking at the maze drawing backwards: > he'd intended the lines the turtle draws to be the *paths*, and *everything > else* was walls. > > Rather than calculating exactly where all the walls would be, to contain a > given path, I just added a background and made the turtle carve out the > path of the maze. > > I also added entrances when the path hits the edge. So now we have a > familiar looking maze: > > [image: Inline images 1] > Instructions: Pick 2 opposite entrances and try to get from one to the > other. > > I wonder if we could let Mike's Physical Turtle loose on this... > > > Tom > > ---------- Forwarded message ---------- > From: Tim Golden > Date: 6 May 2013 19:33 > Subject: Re: [python-uk] more maze > To: python-uk at python.org > > > On 06/05/2013 19:26, Thomas Hunger wrote: > >> Here's an implementation of a random spanning tree where the nodes are >> coordinates on a plane so one can render them later: >> >> https://gist.github.com/teh/**5526976 >> > > Thanks, Thomas. > > For those wondering, last Thursday's London Python Dojo was all about > generating mazes. Most teams didn't have time to fulfil their solution's > entire potential and evidently a few people went home buzzing with ideas or > alternatives. Hence the flow of messages to the list promoting maze > solutions. > > Just so you knew... > > The next Dojo should be on the first Thursday of June, venue to be > announced. Keep an eye on this list and/or follow @ldnpydojo. > > TJG > > > ______________________________**_________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/**mailman/listinfo/python-uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot from 2013-05-07 23:55:52.png Type: image/png Size: 706 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot from 2013-05-12 01:46:09.png Type: image/png Size: 1060 bytes Desc: not available URL: From mike at mikedeplume.com Mon May 13 11:44:33 2013 From: mike at mikedeplume.com (MikedePlume) Date: Mon, 13 May 2013 10:44:33 +0100 Subject: [python-uk] Fwd: more maze In-Reply-To: References: <5187F790.7030402@timgolden.me.uk> Message-ID: <1368438273.2098.23.camel@oberto> On Sun, 2013-05-12 at 10:01 +0100, Tom Viner wrote: > > > I wonder if we could let Mike's Physical Turtle loose on this... Now there's a challenge :-) I was aware from team 1's work that I needed to think about drawing parallel walls. It's just gone back up the todo list. From harry.percival at gmail.com Wed May 15 11:57:16 2013 From: harry.percival at gmail.com (Harry Percival) Date: Wed, 15 May 2013 10:57:16 +0100 Subject: [python-uk] Suggestions / best practices for deployment Message-ID: Dear UK Python chums, some of you probably know I'm writing a book about TDD for O'Reilly. I'm looking for some help with the (first) chapter on deployment. http://www.obeythetestinggoat.com/what-to-say-about-deployment.html What do you use for deployment? Do you have any kind of automated scripts? How do you manage virtualenvs, the database, apache/uwsgi config... What do you think might work as a sort of "best practice lite" for a simple site for beginners? (django, sqlite database, static files) -- ------------------------------ Harry J.W. Percival ------------------------------ Twitter: @hjwp Mobile: +44 (0) 78877 02511 Skype: harry.percival -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim at red56.co.uk Wed May 15 12:28:57 2013 From: tim at red56.co.uk (Tim Diggins) Date: Wed, 15 May 2013 11:28:57 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: Hi Harry (and list) First - I think you should separate out provisioning (setting up the server and all the dependencies and maybe the initial install) from deployment (repeatedly updating the site based on the latest codebase and db migrations etc). Secondly - my experience (which may not be very helpful). In the past, I've used chef to provision configure a rackspace / amazon box, and have customized recipes for python / virtualenv install. I then use a capistrano to deploy the code changes for django . Very little customization is needed for django (as opposed to rails) interestingly, because capistrano is pretty well thought through. Nonetheless it's not a common practice in django/python communities (cause cap was written for rails and is in ruby, I imagine). Would argue that the concept of best practice lite doesn't make sense! It all depends on the constraints and what is the most pressing issue right now. In the context of TDD + python, the best practice lite, might well be (pace your introduction) to use PythonAnywhere or maybe GAE, mightn't it? If you're teaching TDD +ruby, then I'd say best practice lite would be to use heroku. T PS: planning to use PythonAnywhere to teach python to secondary school kids, but I'll maybe contact you separately about that... On 15 May 2013 10:57, Harry Percival wrote: > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for O'Reilly. I'm > looking for some help with the (first) chapter on deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated > scripts? How do you manage virtualenvs, the database, apache/uwsgi > config... What do you think might work as a sort of "best practice lite" > for a simple site for beginners? (django, sqlite database, static files) > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfredo at madewithbyt.es Wed May 15 12:30:46 2013 From: alfredo at madewithbyt.es (Alfredo Aguirre) Date: Wed, 15 May 2013 11:30:46 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: <4AD4005D-487D-4242-A414-CA1DC1CB3E12@madewithbyt.es> Hi Harry, I will get a copy of your book once it is out. I loved the hands-on workshop you gave in London a while ago. Maybe cuisine can help: https://github.com/sebastien/cuisine is a set of scripts to bootstrap applications via fabric. I've been working in a vagrant + puppet setup for a local environment and deployable to Heroku. Please note that this is work in progress and it is far from completed https://github.com/alfredo/django-template-plus But if this is aimed to beginners this would add loads of concepts to the mix. Vagrant, Puppet, VirtualBox and any other under-the-hood bits of the configuration, something like this would feel like a black box. Maybe you could create a simpler setup? Hope this helps. All the best, Alfredo On 15 May 2013, at 10:57, Harry Percival wrote: > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for O'Reilly. I'm looking for some help with the (first) chapter on deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated scripts? How do you manage virtualenvs, the database, apache/uwsgi config... What do you think might work as a sort of "best practice lite" for a simple site for beginners? (django, sqlite database, static files) > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick at nivan.net Wed May 15 12:38:52 2013 From: nick at nivan.net (Nick Murdoch) Date: Wed, 15 May 2013 11:38:52 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: <20130515103852.GD1729@femputer> A very quick rundown of my process: * Custom script to build and send to a packages server: - deleting things like Cythoned files and generated .so files which setup.py likes to try to avoid recompiling even if you want it to. - setup.py build_ext - setup.py sdist -d dist.$TEMP - scp dist.$TEMP/*.tar.gz $packageserver:/path/to/packages * Packages server is basically a www directory with a frilly directory listing for the packages sent to it. A regular directory listing is fine. This is for easy_install/pip's find_links setting. * I deploy things in virtualenvs on Debian/Ubuntu because of the requirement for different versions of packages (CherryPy 2/3 springs to mind) and I don't like setuptools' way of handling multiple versions. However, virtualenv has a big problem with point upgrades to python, so it requires some extra time each time python gets upgraded. I tend to create a dedicated user that owns the virtualenv and that the app will run as. * Currently using easy_install but going to bite the bullet and move to pip at some point... - easy_install -UZf http://$packageserver/mypackages/ MyPackage - add a dud -i (before MyPackage) to avoid polling pypi which can be pointless and (very) slow. * Restart/reload apache (or your service if not using mod_python or mod_wsgi) I much prefer mod_wsgi-compatible frameworks (I think all of them now, but I'm stuck with some old TurboGears 1.x projects!) since reloading apache takes care of everything. My main gripe with mod_wsgi is that it's a bit fiddly to write the deployment.wsgi if you're not familiar with it or your framework doesn't provide a way of generating one. Hope that helps, Nick On Wed, May 15, 2013 at 10:57:16AM +0100, Harry Percival wrote: > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for O'Reilly. I'm > looking for some help with the (first) chapter on deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated scripts? > How do you manage virtualenvs, the database, apache/uwsgi config... What do > you think might work as a sort of "best practice lite" for a simple site > for beginners? (django, sqlite database, static files) > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk From george at ghickman.co.uk Wed May 15 12:34:51 2013 From: george at ghickman.co.uk (George Hickman) Date: Wed, 15 May 2013 11:34:51 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: Hi Harry, I use two methods for deployment - Heroku and Ubuntu VPSs. Heroku is really simple to get going but gets expensive if you want to run a serious production app. However it's probably what I'd recommend for beginners since so much is done for you and they have first class python support. For VPS deploys I use Nginx, Daemontools (for the envdir package), Supervisor, Gunicorn and Postgres (with Django). I usually deploy this setup with a mixture of Salt to build the VPS and Fabric to deploy the code. George On Wed, May 15, 2013 at 10:57 AM, Harry Percival wrote: > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for O'Reilly. I'm > looking for some help with the (first) chapter on deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated > scripts? How do you manage virtualenvs, the database, apache/uwsgi > config... What do you think might work as a sort of "best practice lite" > for a simple site for beginners? (django, sqlite database, static files) > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From deepakgarg.iitg at gmail.com Wed May 15 12:48:34 2013 From: deepakgarg.iitg at gmail.com (Deepak Garg) Date: Wed, 15 May 2013 16:18:34 +0530 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: For small websites ( upto 3 - 4 VMs ) I use fabric. I define small modular tasks that get executed in AWS EC2. I wrote a bunch of scripts around a year before for the same: https://github.com/gargdeepak/django-cloud Its not tested for quite some time and should be read well before being used. For larger deployments I would prefer Puppet (for non AWS deployment) or AWS Elastic Beanstalk. It allows easy management and change in the deployment configurations. Both of the above can be used in a TDD way such that after every step, you would know why exactly the deployment is incomplete / failed and how exactly the output should look like. Thanks, -- Deepak Garg, Phone-no.:+918753985659 Skype-id: deepakgarg.iit Github: https://github.com/gargdeepak LinkedIn: http://in.linkedin.com/in/deepakgargiit Slideshare: http://www.slideshare.net/khinnu4u/presentations On Wed, May 15, 2013 at 3:27 PM, Harry Percival wrote: > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for O'Reilly. I'm > looking for some help with the (first) chapter on deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated > scripts? How do you manage virtualenvs, the database, apache/uwsgi > config... What do you think might work as a sort of "best practice lite" > for a simple site for beginners? (django, sqlite database, static files) > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kris at automationlogic.com Wed May 15 13:10:18 2013 From: kris at automationlogic.com (Kris Saxton) Date: Wed, 15 May 2013 12:10:18 +0100 Subject: [python-uk] Suggestions / best practices for deployment Message-ID: <333B286A-911A-47CC-A6A7-067BE3B36087@automationlogic.com> Hi Harry (and list) Just to expand on what George was saying. Salt (http://saltstack.com) is a remote execution (fabric) and configuration management (puppet, chef) framework written in Python and using ZeroMQ as the transport. I do automation for a living, started with puppet but now prefer to use salt where possible - it's truly magic and catching on fast (it's one of the most active projects on GitHub). I gave a talk about using and extending Salt at the last London Python Meetup, the video is here if you're interested: http://skillsmatter.com/podcast/java-jee/my-experience-of-using-server-management-framework-salt. You can use Salt to spin up cloud-based VMs as well configure them and there are Salt modules for managing the likes of virtualenv, databases and webservers. Looking forward to the TDD book : ) Kris -- Kris Saxton e: kris at automationlogic.com m: +447932834856 t: @KrisSaxton From muhammad.rahman at tangentlabs.co.uk Wed May 15 13:19:54 2013 From: muhammad.rahman at tangentlabs.co.uk (Muhammad Rahman) Date: Wed, 15 May 2013 12:19:54 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: <51936F5A.1040901@tangentlabs.co.uk> Hi Harry, I would use salt(http://docs.saltstack.com/index.html) for provisioning and keep the python specific package to fabric to deploy using pip. Even though salt is able to deploy pip requirements but I think this separation is important. Mustafiz. On 15/05/2013 11:34, George Hickman wrote: > Hi Harry, > > I use two methods for deployment - Heroku and Ubuntu VPSs. > > Heroku is really simple to get going but gets expensive if you want to > run a serious production app. However it's probably what I'd recommend > for beginners since so much is done for you and they have first class > python support. > > For VPS deploys I use Nginx, Daemontools (for the envdir package), > Supervisor, Gunicorn and Postgres (with Django). I usually deploy this > setup with a mixture of Salt to build the VPS and Fabric to deploy the > code. > > George > > > On Wed, May 15, 2013 at 10:57 AM, Harry Percival > > wrote: > > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for > O'Reilly. I'm looking for some help with the (first) chapter on > deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated > scripts? How do you manage virtualenvs, the database, apache/uwsgi > config... What do you think might work as a sort of "best practice > lite" for a simple site for beginners? (django, sqlite database, > static files) > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk -- Muhammad Rahman Developer Tangent Labs 84 - 86 Great Portland Street London, W1W 7NR T: +44 (0)207 462 6150 (Office) F: +44 (0)207 462 6111 W: www.tangentlabs.co.uk -------------- next part -------------- An HTML attachment was scrubbed... URL: From renesd at gmail.com Wed May 15 13:28:29 2013 From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=) Date: Wed, 15 May 2013 13:28:29 +0200 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: Every other reply completely failed at TDD! Write the test first to see if the URL is responding(oh, maybe test if the domain name is registered and trademarked first?)? Also, there's no best practices for python deployment... just a bunch of random hacks. You'll learn that moving to different projects... WHO ALL DEPLOY DIFFERENTLY. Maybe it has something to do with different user stories having different requirements for deployment... hrmm. If you can write a coherent, short, and useful chapter on this topic, I'd be very impressed! Joking aside... buildout, and creating packages(.deb/.rpm etc) should be mentioned... "Buildout is an exceedingly civilized way to develop an app." --Jacob Kaplan-Moss, creator of Django -------------- next part -------------- An HTML attachment was scrubbed... URL: From stestagg at gmail.com Wed May 15 13:28:59 2013 From: stestagg at gmail.com (Stestagg) Date: Wed, 15 May 2013 12:28:59 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: <51936F5A.1040901@tangentlabs.co.uk> References: <51936F5A.1040901@tangentlabs.co.uk> Message-ID: To add a slightly different angle to this, whatever deployment solution you use, make sure it is fully automated, and then hook it into you CI system. Deployment, disaster recovery etc. are made so much simpler if you're thinking about it from the start rather than just before release. One way to enforce this is to start running the process regularly. My personal preference for this is to use snapshotted virtualbox VMs to deploy the service/app to a clean OS install, and the run the tests, after every commit (or on a schedule). This way, you always know what dependencies you need, and you're forced to make things as seamless as possible. The same goes for testing upgrades. Set up a VM with version X and set up a job to: Upgrade > Test > Check > Rollback regularly. Thanks Steve On Wed, May 15, 2013 at 12:19 PM, Muhammad Rahman < muhammad.rahman at tangentlabs.co.uk> wrote: > Hi Harry, > > I would use salt(http://docs.saltstack.com/index.html) for provisioning > and keep the python specific package to fabric to deploy using pip. > > Even though salt is able to deploy pip requirements but I think this > separation is important. > > Mustafiz. > > > On 15/05/2013 11:34, George Hickman wrote: > > Hi Harry, > > I use two methods for deployment - Heroku and Ubuntu VPSs. > > Heroku is really simple to get going but gets expensive if you want to > run a serious production app. However it's probably what I'd recommend for > beginners since so much is done for you and they have first class python > support. > > For VPS deploys I use Nginx, Daemontools (for the envdir package), > Supervisor, Gunicorn and Postgres (with Django). I usually deploy this > setup with a mixture of Salt to build the VPS and Fabric to deploy the code. > > George > > > On Wed, May 15, 2013 at 10:57 AM, Harry Percival > wrote: > >> Dear UK Python chums, >> >> some of you probably know I'm writing a book about TDD for O'Reilly. >> I'm looking for some help with the (first) chapter on deployment. >> >> http://www.obeythetestinggoat.com/what-to-say-about-deployment.html >> >> What do you use for deployment? Do you have any kind of automated >> scripts? How do you manage virtualenvs, the database, apache/uwsgi >> config... What do you think might work as a sort of "best practice lite" >> for a simple site for beginners? (django, sqlite database, static files) >> >> -- >> ------------------------------ >> Harry J.W. Percival >> ------------------------------ >> Twitter: @hjwp >> Mobile: +44 (0) 78877 02511 <%2B44%20%280%29%2078877%2002511> >> Skype: harry.percival >> >> _______________________________________________ >> python-uk mailing list >> python-uk at python.org >> http://mail.python.org/mailman/listinfo/python-uk >> >> > > > _______________________________________________ > python-uk mailing listpython-uk at python.orghttp://mail.python.org/mailman/listinfo/python-uk > > > -- > > Muhammad Rahman > Developer > > Tangent Labs > 84 - 86 Great Portland Street > London, W1W 7NR > > T: +44 (0)207 462 6150 (Office) > F: +44 (0)207 462 6111 > W: www.tangentlabs.co.uk > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matth at netsight.co.uk Wed May 15 13:00:16 2013 From: matth at netsight.co.uk (Matt Hamilton) Date: Wed, 15 May 2013 12:00:16 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: On 15 May 2013, at 10:57, Harry Percival wrote: > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for O'Reilly. I'm looking for some help with the (first) chapter on deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated scripts? How do you manage virtualenvs, the database, apache/uwsgi config... What do you think might work as a sort of "best practice lite" for a simple site for beginners? (django, sqlite database, static files) We use zc.buildout for all our deployment stuff, as it can handle not just python things, but also other stuff too (e.g. setting up a Solr server, or varnish proxy, etc) and allows multiple buildout files with inheritances between them. ie. we have a base.cfg, a dev.cfg, a staging.cfg and a live.cfg, the latter three build upon the first. Live might set up a cluster of Zope instances, an haproxy instance and varnish. dev.cfg might set up debuggers and extra reporting etc. To take this further, the Plone community does a lot of CI testing and buildout helps with this as you could have a travis.cfg which might set up the environment for Travis. A good set of slides on this: http://www.slideshare.net/zupo/travis-ci-fun-and-easy-ci-for-your-plone-packages Or, for instance, we use Jenkins in-house and one of our projects, a Pyramid project has a specific buildout file for use with jenkins which uses sqllite instead of Postgres, and sets up some default content for the tests to run against. -Matt -- Matt Hamilton, Technical Director Netsight Internet Solutions Limited http://www.netsight.co.uk/matth Tel: 0117 90 90 90 1 Ext. 15 Registered in England No. 3892180 Registered office: 40 Berkeley Square, Clifton, Bristol, BS8 1HU From andy at reportlab.com Wed May 15 13:54:46 2013 From: andy at reportlab.com (Andy Robinson) Date: Wed, 15 May 2013 12:54:46 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: Harry, are you anywhere near London? If so, you would be welcome to pop into ReportLab some time and we could tell you endless war stories about deployment. I remember giving a talk on the subject at EuroPython in 2003 and basically scaring the hell out of all the small startups present about how much of their time would be wasted on this in coming years. Each team evolves their own practices, and a lot depends on whether you have one team working on one project or (like us) you have anywhere from 10-20 systems and are battling to keep them all deployed in a consistent way, as all the other tools around you (Python, pip, Django, wsgi etc) all insist on continuing to evolve around you. One big headache is that pip almost-works but can't handle a few packages (numpy). We have four key and separate areas: 1. How do you set up a server? 2. How should a developer / tester set up their new machine? 3. How to install a new project on a server? 4. How to set up a local development environment for a project? (which often involves automatically copying down recent snapshots of live data and files, to reproduce the live environment). Probably worth a look at the recent "Two Scoops of Django" book which has some good points on best practices. On 15 May 2013 10:57, Harry Percival wrote: > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for O'Reilly. I'm > looking for some help with the (first) chapter on deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated scripts? > How do you manage virtualenvs, the database, apache/uwsgi config... What do > you think might work as a sort of "best practice lite" for a simple site for > beginners? (django, sqlite database, static files) > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > -- Andy Robinson Managing Director ReportLab Europe Ltd. Thornton House, Thornton Road, Wimbledon, London SW19 4NG, UK Tel +44-20-8405-6420 From ed.hartley at gmail.com Wed May 15 15:10:38 2013 From: ed.hartley at gmail.com (E Hartley) Date: Wed, 15 May 2013 14:10:38 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> One point that's worth mentioning in this context: If you rely on building native libraries for custom Python packages of your own or those created by third parties consider avoiding the automatic systems that do security updates in the background. We had a sysadmin at point who insisted on this and had a series of package breakages as the library dependencies broke. HTH Ed E Hartley ed.hartley at gmail.com On 15 May 2013, at 10:57, Harry Percival wrote: > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for O'Reilly. I'm looking for some help with the (first) chapter on deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated scripts? How do you manage virtualenvs, the database, apache/uwsgi config... What do you think might work as a sort of "best practice lite" for a simple site for beginners? (django, sqlite database, static files) > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk -------------- next part -------------- An HTML attachment was scrubbed... URL: From harry.percival at gmail.com Thu May 16 10:02:07 2013 From: harry.percival at gmail.com (Harry Percival) Date: Thu, 16 May 2013 09:02:07 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> Message-ID: Hi everyone, thanks for some helpful thoughts! @Tim, thanks for pointing out the distinction between provisioning and deployment, I think that's helped to clarify things in my head a little... @Alfredo, @Kris, and anyone else that's curious, you can already buy/read the book, plug plug. I'm 6 chapters in and the 7th is the one on deployment which I'm writing now... www.obeythetestinggoat.com @Andy, I am in London, thanks for the invite, will drop you a line. Thanks everyone for the suggestions re: salt and zc.buildout, definitely want to check those out, I think they may be overkill for this stage in the book. Cuisine actually looks pretty cool too, might be just the kind of lightweight wrapper over fabric that we could use at work (currently ~4000 lines of code based on fabric currently in the deploys folder of the codebase for PythonAnywhere!) Overall, it's both reassuring and depressing to hear that that there's no single accepted way to do it! For my use case, I think it comes down to keeping things as simple as possible. I don't have to sort out all of the complexities of deployment, or even most of them. This is an early chapter, and I can come back and introduce more subtlety later. For the sake of argument, I'll pick a hosting solution -- say AWS, or whichever VPS. And let's say also that you're using Apache. There's a lot of it around. We then start with "provisioning" -- that's getting a server up and running with Apache installed. Let's say that's something you don't automate, for now. Moving onto "deployment", that could cover: - uploading / updating your source code on the server - writing (or overwriting) the apache httpd.conf / sites-available entry - making sure you're hooked up to the (right) database - checking static files are working. That sounds like a couple of fairly simple fabric scripts, and a simple set of functional tests for checking static files and database work (and that you haven't killed any old data). You can do a deploy to staging, run your tests, and then have confidence that your deploy to live will be fine. I'm not sure if I want to bring in virtualenvs at this stage. If the assumption is that things are on the same server, testing staging means you can be sure live will work. Of course, the day you need to upgrade a package, you have to switch to using virtualenvs, but that's something I could bring in later in the book. OTOH, if I want to acknowledge the fact that most people would be better off using Heroku or PythonAnywere or one of the other PaaSes, maybe I should mention virtualenvs, because they're a bit of a sine qua non... hmmm hp PS - loving the new gmail "switch to pop-over" feature for email replies to this thread... On 15 May 2013 14:10, E Hartley wrote: > One point that's worth mentioning in this context: > If you rely on building native libraries for custom Python packages of > your own or those created by third parties consider avoiding the automatic > systems that do security updates in the background. > We had a sysadmin at point who insisted on this and had a series of > package breakages as the library dependencies broke. > HTH > Ed > > E Hartley > ed.hartley at gmail.com > > > > > > On 15 May 2013, at 10:57, Harry Percival wrote: > > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for O'Reilly. I'm > looking for some help with the (first) chapter on deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated > scripts? How do you manage virtualenvs, the database, apache/uwsgi > config... What do you think might work as a sort of "best practice lite" > for a simple site for beginners? (django, sqlite database, static files) > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > -- ------------------------------ Harry J.W. Percival ------------------------------ Twitter: @hjwp Mobile: +44 (0) 78877 02511 Skype: harry.percival -------------- next part -------------- An HTML attachment was scrubbed... URL: From matth at netsight.co.uk Thu May 16 10:43:47 2013 From: matth at netsight.co.uk (Matt Hamilton) Date: Thu, 16 May 2013 09:43:47 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> Message-ID: On 16 May 2013, at 09:02, Harry Percival wrote: > Overall, it's both reassuring and depressing to hear that that there's no single accepted way to do it! *snip* because?. > We then start with "provisioning" -- that's getting a server up and running with Apache installed. Let's say that's something you don't automate, for now. > > Moving onto "deployment", that could cover: > - uploading / updating your source code on the server > - writing (or overwriting) the apache httpd.conf / sites-available entry > - making sure you're hooked up to the (right) database > - checking static files are working. > > That sounds like a couple of fairly simple fabric scripts, and a simple set of functional tests for checking static files and database work (and that you haven't killed any old data). You can do a deploy to staging, run your tests, and then have confidence that your deploy to live will be fine. That *your* use case above is not the same as *my* use case ;) So no doubt people are going to have different priorities and hence use different tools dependant on those priorities. OK, yes there *are* too many tools, and I fear many of them are due to not-invented-here syndrome. But still?. there will never be *one* single accepted way of doing things, and we all do different things. -Matt -- Matt Hamilton, Technical Director Netsight Internet Solutions Limited http://www.netsight.co.uk/matth Tel: 0117 90 90 90 1 Ext. 15 Registered in England No. 3892180 Registered office: 40 Berkeley Square, Clifton, Bristol, BS8 1HU From tim at red56.co.uk Thu May 16 12:22:08 2013 From: tim at red56.co.uk (Tim Diggins) Date: Thu, 16 May 2013 11:22:08 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> Message-ID: Harry - I think if you're teaching best practices on python and testing, you've got to teach virtualenv - maybe not for deployment (cause better to have separate VMs for staging and production) but for development and testing, really important (unless you do everything in a vm, like vagrant or similar). Can't imagine how you could set up an effective CI server without virtualenv, for example. Tim PS at the risk of starting a flame war: "zc.buildout"? --shudder-- Feels like there are two different python communities sometimes, those that use zope-derived rather java-flavoured tools and mindset, and the rest. On 16 May 2013 09:43, Matt Hamilton wrote: > > On 16 May 2013, at 09:02, Harry Percival wrote: > > > Overall, it's both reassuring and depressing to hear that that there's > no single accepted way to do it! > > *snip* > > because?. > > > We then start with "provisioning" -- that's getting a server up and > running with Apache installed. Let's say that's something you don't > automate, for now. > > > > Moving onto "deployment", that could cover: > > - uploading / updating your source code on the server > > - writing (or overwriting) the apache httpd.conf / sites-available entry > > - making sure you're hooked up to the (right) database > > - checking static files are working. > > > > That sounds like a couple of fairly simple fabric scripts, and a simple > set of functional tests for checking static files and database work (and > that you haven't killed any old data). You can do a deploy to staging, run > your tests, and then have confidence that your deploy to live will be fine. > > That *your* use case above is not the same as *my* use case ;) > > So no doubt people are going to have different priorities and hence use > different tools dependant on those priorities. > > OK, yes there *are* too many tools, and I fear many of them are due to > not-invented-here syndrome. But still?. there will never be *one* single > accepted way of doing things, and we all do different things. > > -Matt > > -- > Matt Hamilton, Technical Director > Netsight Internet Solutions Limited > http://www.netsight.co.uk/matth > Tel: 0117 90 90 90 1 Ext. 15 > > Registered in England No. 3892180 > Registered office: 40 Berkeley Square, Clifton, Bristol, BS8 1HU > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sparks.m at gmail.com Thu May 16 12:32:50 2013 From: sparks.m at gmail.com (Michael) Date: Thu, 16 May 2013 11:32:50 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: While we're on topic - what would people say is best practice for testing of a web API in a way that slots cleanly into a CI setup? Lots of different options out there, hence the q. (I realise this is also a bit like saying how long is a piece of string) Michael. On 15 May 2013 10:57, Harry Percival wrote: > Dear UK Python chums, > > some of you probably know I'm writing a book about TDD for O'Reilly. I'm > looking for some help with the (first) chapter on deployment. > > http://www.obeythetestinggoat.com/what-to-say-about-deployment.html > > What do you use for deployment? Do you have any kind of automated > scripts? How do you manage virtualenvs, the database, apache/uwsgi > config... What do you think might work as a sort of "best practice lite" > for a simple site for beginners? (django, sqlite database, static files) > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matth at netsight.co.uk Thu May 16 12:47:07 2013 From: matth at netsight.co.uk (Matt Hamilton) Date: Thu, 16 May 2013 11:47:07 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> Message-ID: <37A430C4-E7DB-411E-A976-764530733531@netsight.co.uk> On 16 May 2013, at 11:22, Tim Diggins wrote: > PS at the risk of starting a flame war: "zc.buildout"? --shudder-- > Feels like there are two different python communities sometimes, those that use zope-derived rather java-flavoured tools and mindset, and the rest. I don't want to start a flamewar either, but I am interested in what exactly makes you shudder about zc.buildout? I agree with you about the two separate communities?well to a degree. When Zope started way back when, there was no best practise, and so it had to create its own path. There were a lot of mistakes along the way, and a lot of lessons learnt. But there were a lot of things that were way more advanced than anything out there at the time. There was no easy_install, pip, eggs? even PyPI (the cheeseshop) was in its infancy. Zope2 was a massive beast with quite a bit of momentum and it took quite some time to re-align it with the best practise as it emerged. But these days, it is very 'pythonic' and fits in with the rest of the community pretty will I think. I think the people that generally run screaming when the hear anything mentioned that starts with a 'z' are those that experienced Zope in its teenage years when it was transitioning to follow best practice. The Zope community was doing big web application deployments over a decade ago and needed tools to manage deployment, configuration management, etc and so had to build those tools as the term 'devops' had not even been invented yet. A lot of knowledge, experience, pain, and ultimately success has come out of that community, and I always feel a bit sad when it is written off by people who don't like the smell/flavour of it. The Pyramid developers took their knowledge and experience of Zope and learnt from its shortcomings and successes and went on to create something that really is truly awesome. I wish there was more of that in the Python community. -Matt -- Matt Hamilton, Technical Director Netsight Internet Solutions Limited http://www.netsight.co.uk/matth Tel: 0117 90 90 90 1 Ext. 15 Registered in England No. 3892180 Registered office: 40 Berkeley Square, Clifton, Bristol, BS8 1HU From stestagg at gmail.com Thu May 16 12:52:31 2013 From: stestagg at gmail.com (Stestagg) Date: Thu, 16 May 2013 11:52:31 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: <37A430C4-E7DB-411E-A976-764530733531@netsight.co.uk> References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <37A430C4-E7DB-411E-A976-764530733531@netsight.co.uk> Message-ID: The Zope 'brand' got trashed back in the bad old days. If things have truly improved, then the sensible thing to do would be to release the new code in a way that has no obvious links to the name 'Zope', and let that stand on its own merits. Another factor here, is that the current trend is towards loosely coupled, modular systems, look at flask or (to an extent) django. The feeling of having to 'buy in' to an ecosystem goes against the python ethos, in my opinion. Steve On Thu, May 16, 2013 at 11:47 AM, Matt Hamilton wrote: > > On 16 May 2013, at 11:22, Tim Diggins wrote: > > > PS at the risk of starting a flame war: "zc.buildout"? --shudder-- > > Feels like there are two different python communities sometimes, those > that use zope-derived rather java-flavoured tools and mindset, and the rest. > > I don't want to start a flamewar either, but I am interested in what > exactly makes you shudder about zc.buildout? > > I agree with you about the two separate communities?well to a degree. When > Zope started way back when, there was no best practise, and so it had to > create its own path. There were a lot of mistakes along the way, and a lot > of lessons learnt. But there were a lot of things that were way more > advanced than anything out there at the time. There was no easy_install, > pip, eggs? even PyPI (the cheeseshop) was in its infancy. > > Zope2 was a massive beast with quite a bit of momentum and it took quite > some time to re-align it with the best practise as it emerged. > > But these days, it is very 'pythonic' and fits in with the rest of the > community pretty will I think. I think the people that generally run > screaming when the hear anything mentioned that starts with a 'z' are those > that experienced Zope in its teenage years when it was transitioning to > follow best practice. > > The Zope community was doing big web application deployments over a decade > ago and needed tools to manage deployment, configuration management, etc > and so had to build those tools as the term 'devops' had not even been > invented yet. A lot of knowledge, experience, pain, and ultimately success > has come out of that community, and I always feel a bit sad when it is > written off by people who don't like the smell/flavour of it. > > The Pyramid developers took their knowledge and experience of Zope and > learnt from its shortcomings and successes and went on to create something > that really is truly awesome. I wish there was more of that in the Python > community. > > -Matt > > -- > Matt Hamilton, Technical Director > Netsight Internet Solutions Limited > http://www.netsight.co.uk/matth > Tel: 0117 90 90 90 1 Ext. 15 > > Registered in England No. 3892180 > Registered office: 40 Berkeley Square, Clifton, Bristol, BS8 1HU > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matth at netsight.co.uk Thu May 16 13:06:44 2013 From: matth at netsight.co.uk (Matt Hamilton) Date: Thu, 16 May 2013 12:06:44 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <37A430C4-E7DB-411E-A976-764530733531@netsight.co.uk> Message-ID: On 16 May 2013, at 11:52, Stestagg wrote: > The Zope 'brand' got trashed back in the bad old days. If things have truly improved, then the sensible thing to do would be to release the new code in a way that has no obvious links to the name 'Zope', and let that stand on its own merits. Well, that has sort of happened, Zope 3 got renamed to BlueBream. But most of the Zope Toolkit stuff still has the name in it. But alas people can't see past the name, which is a shame. > Another factor here, is that the current trend is towards loosely coupled, modular systems, look at flask or (to an extent) django. The feeling of having to 'buy in' to an ecosystem goes against the python ethos, in my opinion. Django? You are kidding me right? They have taken the same monolithic approach that Zope took. Hence my comments about Pyramid, they learnt from that and it is highly modular and you can use the bits you want (e.g. choose your tempting system, choose your ORM or ZODB etc). If you look at Plone, which is one of the two major systems still built on Zope2 (the other being Zenoss) then you will see that Plone is highly modular. A base Plone 4.3 install is comprised of 247 eggs, 54 of which are in the zope.* namespace Probably why we need Buildout! ;) 'Zope2' which was the old Zope application server is just 1 of those 247 eggs. -Matt -- Matt Hamilton, Technical Director Netsight Internet Solutions Limited http://www.netsight.co.uk/matth Tel: 0117 90 90 90 1 Ext. 15 Registered in England No. 3892180 Registered office: 40 Berkeley Square, Clifton, Bristol, BS8 1HU From andy at reportlab.com Thu May 16 17:46:05 2013 From: andy at reportlab.com (Andy Robinson) Date: Thu, 16 May 2013 16:46:05 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> Message-ID: Speaking as a relatively obsolete dinosaur, I would suggest that if you are going to discuss specific deployment practices, you start with the most fundamental ones: SSH, the unix shell and so on. We have had issues over the years with people coming in and introducing sexy new deployment tools, but ultimately they all just run unix commands. Anyone managing a web application in the non-microsoft world is ultimately depending on this. Some key skills (assuming a Linux/Mac/Unix-ish environment): - know about SSH keys and logging into remote machines - know the basics of at least one command line editor (e.g. vi) - basic shell knowledge: environment variables, testing for existence of files and directories etc - how to interact with your database from the command line, if you use one (including dump and restore) - how your web server works: starting, stopping, configuration files, where log files live and how it talks to Python Fabric may be useful if you want to control half a dozen machines from your desktop, and it might add a lot of value if you want to control a hundred of them. But to update one server, you deploy by logging into it and then running commands or short scripts. For example, we have a 'demo site' we rebuild pretty often that uses Django, MYSQL, Celery and a few other things. It runs on plain vanilla Ubuntu machines we build ourselves. The sequence is... 1. Log in via SSH 2. CD to correct directory 3. activate virtual environment 4. stop any celery worker processes 5. stop web server processes (* in our setup, we leave Apache running) 6. pull latest code from mercurial - both the app, and 3-4 libraries it depends on 7. run a management command to rebuild the database 8. run a smallish in-place test suite 9. restart celery workers 10.restart web server 11. log out All of this after the login and CD can be handled by a shell script on the path of the server, so you can just run a command called something like ./update_server More realistically, we tend to end up with a management shell script called 'server' with a bunch of commands/arguments like 'stop / start / restart / update-code-in-staging / copy-live-data-to-staging / run-health-checks / swap-live-and-staging' and so on. SSH can execute remote commands like this just fine with the right arguments, if actually logging in is too tedious. Production sites are complex and all different. You might want to do instantaneous swaps from live to staging (and be able to back out fast if stuff goes wrong); to switch DNS so the world is looking at another server while you update one; you might have large databases to copy or migrate that need significant time; it may or may not be acceptable to lose sessions and have downtime; and so on. It takes less time to learn the fundamentals than you will spend debugging why your fancy new deployment tool stopped working after some Python dependency upgrade somewhere. And it is less likely that your new hires will disagree if you stick with the lowest common denominator. If you already know the fundamentals and make an informed decision to use a popular deployment tool, that's fine. Just take the time to write down why you use it in your docs so people will know if its no longer appropriate one day. --- So, my 2p worth is that in the book you might want to show a Linux/Apache setup, discuss what kind of scripts ought to exist on the box for managing it, discuss concerns you MIGHT need to address during deployment, and tell people to automate it. Then point out that there are many popular higher-level tools. - Andy From julius at yplanapp.com Thu May 16 18:26:41 2013 From: julius at yplanapp.com (=?UTF-8?Q?Julius_=C5=A0=C4=97poraitis?=) Date: Thu, 16 May 2013 17:26:41 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> Message-ID: I would like to second to Andy Robinson here - try to use less new sexy tools and try to use an existing infrastructure. For example, how we build & deploy web services*** at YPlan: 1. A bash script is executed (by Jenkins) on a build machine, which generates an rpm package with [virtualenv + code] embedded and uploads that rpm file to a private S3 bucket (rpm repository). 2. Deployment boils down to having an on-boot hook, that does "yum install " (obviously the server has to be prepared/preconfigured to expose the new service via nginx or Apache). *** background jobs involve a little different approach. The best bit is that if you are on AWS you can write a CloudFormation script, that says "launch autoscaling group, with instances having launch configuration X and attach them to load balancer Y". This perfectly exploits couple of AWS services: 1. S3 for highly available and secure package storage servers (from top of my head - S3 provides 5 nines of availability and an ability for very granular access policy). 2. With CloudFormation you can have your server configuration in version control (that's always a good thing). 3. You are using EC2 as a utility (servers are "recycled" with each deployment) 4. You can always roll back quickly (keep an old version running for a while). 5. You can always recreate any production environment from the past (checkout a tag and upload that cloudformation script). Of course, this will not/does not work for everyone, but it worked for us and I find this set-up very easy to manage while exploiting some of pre-existing tools and infrastructure to our advantage. As a reference/reading material about deployment pipeline best practices I can recommend: "Continuous Delivery" ( http://www.amazon.com/gp/product/0321601912) from ThoughtWorks guys - it is big, sometimes repetitive, but it is good reference material. On Thu, May 16, 2013 at 4:46 PM, Andy Robinson wrote: > Speaking as a relatively obsolete dinosaur, I would suggest that if > you are going to discuss specific deployment practices, you start with > the most fundamental ones: SSH, the unix shell and so on. > > We have had issues over the years with people coming in and > introducing sexy new deployment tools, but ultimately they all just > run unix commands. Anyone managing a web application in the > non-microsoft world is ultimately depending on this. Some key skills > (assuming a Linux/Mac/Unix-ish environment): > - know about SSH keys and logging into remote machines > - know the basics of at least one command line editor (e.g. vi) > - basic shell knowledge: environment variables, testing for existence > of files and directories etc > - how to interact with your database from the command line, if you use > one (including dump and restore) > - how your web server works: starting, stopping, configuration files, > where log files live and how it talks to Python > > Fabric may be useful if you want to control half a dozen machines from > your desktop, and it might add a lot of value if you want to control a > hundred of them. But to update one server, you deploy by logging into > it and then running commands or short scripts. > > For example, we have a 'demo site' we rebuild pretty often that uses > Django, MYSQL, Celery and a few other things. It runs on plain > vanilla Ubuntu machines we build ourselves. The sequence is... > > 1. Log in via SSH > 2. CD to correct directory > 3. activate virtual environment > 4. stop any celery worker processes > 5. stop web server processes (* in our setup, we leave Apache running) > 6. pull latest code from mercurial - both the app, and 3-4 libraries > it depends on > 7. run a management command to rebuild the database > 8. run a smallish in-place test suite > 9. restart celery workers > 10.restart web server > 11. log out > > All of this after the login and CD can be handled by a shell script on > the path of the server, so you can just run a command called something > like > ./update_server > > More realistically, we tend to end up with a management shell script > called 'server' with a bunch of commands/arguments like 'stop / start > / restart / update-code-in-staging / copy-live-data-to-staging / > run-health-checks / swap-live-and-staging' and so on. SSH can execute > remote commands like this just fine with the right arguments, if > actually logging in is too tedious. > > Production sites are complex and all different. You might want to do > instantaneous swaps from live to staging (and be able to back out fast > if stuff goes wrong); to switch DNS so the world is looking at another > server while you update one; you might have large databases to copy or > migrate that need significant time; it may or may not be acceptable to > lose sessions and have downtime; and so on. > > > It takes less time to learn the fundamentals than you will spend > debugging why your fancy new deployment tool stopped working after > some Python dependency upgrade somewhere. And it is less likely that > your new hires will disagree if you stick with the lowest common > denominator. > > If you already know the fundamentals and make an informed decision to > use a popular deployment tool, that's fine. Just take the time to > write down why you use it in your docs so people will know if its no > longer appropriate one day. > > --- > > So, my 2p worth is that in the book you might want to show a > Linux/Apache setup, discuss what kind of scripts ought to exist on the > box for managing it, discuss concerns you MIGHT need to address during > deployment, and tell people to automate it. Then point out that > there are many popular higher-level tools. > > - Andy > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > -- Julius Seporaitis | Senior Software Engineer, YPlan m: +44 7593 678 871 No plan for tonight? Download YPlan London on App Store -------------- next part -------------- An HTML attachment was scrubbed... URL: From ed.hartley at gmail.com Thu May 16 18:57:06 2013 From: ed.hartley at gmail.com (Edward Hartley) Date: Thu, 16 May 2013 17:57:06 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> Message-ID: <62D9F1D6-A04B-408E-9B86-9CE28E46FE0A@gmail.com> +1 from another dinosaur Ed On 16 May 2013, at 16:46, Andy Robinson wrote: > Speaking as a relatively obsolete dinosaur, I would suggest that if > you are going to discuss specific deployment practices, you start with > the most fundamental ones: SSH, the unix shell and so on. > > We have had issues over the years with people coming in and > introducing sexy new deployment tools, but ultimately they all just > run unix commands. Anyone managing a web application in the > non-microsoft world is ultimately depending on this. Some key skills > (assuming a Linux/Mac/Unix-ish environment): > - know about SSH keys and logging into remote machines > - know the basics of at least one command line editor (e.g. vi) > - basic shell knowledge: environment variables, testing for existence > of files and directories etc > - how to interact with your database from the command line, if you use > one (including dump and restore) > - how your web server works: starting, stopping, configuration files, > where log files live and how it talks to Python > > Fabric may be useful if you want to control half a dozen machines from > your desktop, and it might add a lot of value if you want to control a > hundred of them. But to update one server, you deploy by logging into > it and then running commands or short scripts. > > For example, we have a 'demo site' we rebuild pretty often that uses > Django, MYSQL, Celery and a few other things. It runs on plain > vanilla Ubuntu machines we build ourselves. The sequence is... > > 1. Log in via SSH > 2. CD to correct directory > 3. activate virtual environment > 4. stop any celery worker processes > 5. stop web server processes (* in our setup, we leave Apache running) > 6. pull latest code from mercurial - both the app, and 3-4 libraries > it depends on > 7. run a management command to rebuild the database > 8. run a smallish in-place test suite > 9. restart celery workers > 10.restart web server > 11. log out > > All of this after the login and CD can be handled by a shell script on > the path of the server, so you can just run a command called something > like > ./update_server > > More realistically, we tend to end up with a management shell script > called 'server' with a bunch of commands/arguments like 'stop / start > / restart / update-code-in-staging / copy-live-data-to-staging / > run-health-checks / swap-live-and-staging' and so on. SSH can execute > remote commands like this just fine with the right arguments, if > actually logging in is too tedious. > > Production sites are complex and all different. You might want to do > instantaneous swaps from live to staging (and be able to back out fast > if stuff goes wrong); to switch DNS so the world is looking at another > server while you update one; you might have large databases to copy or > migrate that need significant time; it may or may not be acceptable to > lose sessions and have downtime; and so on. > > > It takes less time to learn the fundamentals than you will spend > debugging why your fancy new deployment tool stopped working after > some Python dependency upgrade somewhere. And it is less likely that > your new hires will disagree if you stick with the lowest common > denominator. > > If you already know the fundamentals and make an informed decision to > use a popular deployment tool, that's fine. Just take the time to > write down why you use it in your docs so people will know if its no > longer appropriate one day. > > --- > > So, my 2p worth is that in the book you might want to show a > Linux/Apache setup, discuss what kind of scripts ought to exist on the > box for managing it, discuss concerns you MIGHT need to address during > deployment, and tell people to automate it. Then point out that > there are many popular higher-level tools. > > - Andy > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk From salim.fadhley at baml.com Fri May 17 16:06:14 2013 From: salim.fadhley at baml.com (Fadhley, Salim) Date: Fri, 17 May 2013 14:06:14 +0000 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: Message-ID: In a previous job I was asked to set up CI & (almost) automatic deployment for a financial pricing library + UI which had a large multi-step build and an annoying test-routine - which included loads of regression tests which had been written in MS-Excel. This was a desktop application used by a few hundred commodities traders. The deployment was an utter nuisance because of the number of components involved (eggs, XLLs, DLLs, configuration files). The structure of the project was set in stone. My job was to make the test & deployment process better. One of the things I built to make this happen was a Python API for easily getting at stuff built in Jenkins. I'd set up a pipeline of jobs which might build and then test each component of the build. Furthermore, the API could reliably query Jenkins, and perform tasks such as retrieving a set of artifacts from the most recent run with no test-failures. I was able to use my JenkinsAPI to fetch the component built in the previous stage (stored as a Jenkins Artifact) and then quickly deploy that into the into the Jenkins workspace downstream. You can get it here: https://github.com/salimfadhley/jenkinsapi ---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. From mal at egenix.com Fri May 17 16:39:20 2013 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 17 May 2013 16:39:20 +0200 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> Message-ID: <51964118.4040402@egenix.com> On 16.05.2013 17:46, Andy Robinson wrote: > Speaking as a relatively obsolete dinosaur, I would suggest that if > you are going to discuss specific deployment practices, you start with > the most fundamental ones: SSH, the unix shell and so on. > > We have had issues over the years with people coming in and > introducing sexy new deployment tools, but ultimately they all just > run unix commands. Anyone managing a web application in the > non-microsoft world is ultimately depending on this. Some key skills > (assuming a Linux/Mac/Unix-ish environment): > - know about SSH keys and logging into remote machines > - know the basics of at least one command line editor (e.g. vi) > - basic shell knowledge: environment variables, testing for existence > of files and directories etc > - how to interact with your database from the command line, if you use > one (including dump and restore) > - how your web server works: starting, stopping, configuration files, > where log files live and how it talks to Python > > Fabric may be useful if you want to control half a dozen machines from > your desktop, and it might add a lot of value if you want to control a > hundred of them. But to update one server, you deploy by logging into > it and then running commands or short scripts. > > For example, we have a 'demo site' we rebuild pretty often that uses > Django, MYSQL, Celery and a few other things. It runs on plain > vanilla Ubuntu machines we build ourselves. The sequence is... > > 1. Log in via SSH > 2. CD to correct directory > 3. activate virtual environment > 4. stop any celery worker processes > 5. stop web server processes (* in our setup, we leave Apache running) > 6. pull latest code from mercurial - both the app, and 3-4 libraries > it depends on > 7. run a management command to rebuild the database > 8. run a smallish in-place test suite > 9. restart celery workers > 10.restart web server > 11. log out > > All of this after the login and CD can be handled by a shell script on > the path of the server, so you can just run a command called something > like > ./update_server > > More realistically, we tend to end up with a management shell script > called 'server' with a bunch of commands/arguments like 'stop / start > / restart / update-code-in-staging / copy-live-data-to-staging / > run-health-checks / swap-live-and-staging' and so on. SSH can execute > remote commands like this just fine with the right arguments, if > actually logging in is too tedious. > > Production sites are complex and all different. You might want to do > instantaneous swaps from live to staging (and be able to back out fast > if stuff goes wrong); to switch DNS so the world is looking at another > server while you update one; you might have large databases to copy or > migrate that need significant time; it may or may not be acceptable to > lose sessions and have downtime; and so on. > > > It takes less time to learn the fundamentals than you will spend > debugging why your fancy new deployment tool stopped working after > some Python dependency upgrade somewhere. And it is less likely that > your new hires will disagree if you stick with the lowest common > denominator. Fully agreed. The new devops tools are nice when it comes to managing farms of VMs, where each VM is setup in more or less the same way, and you want to reduce repetitive tasks, but for a typical setup with just a few VMs/servers it'll take you longer to write and (most importantly) test your devops scripts than it would to write a few scripts that you can run via SSH on the servers. No matter how smart you make your devops scripts, Murphy's law is inevitably going to strike and humans are so much better at parsing weird intermittent error messages than machines ... still, after all these years :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 17 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2013-05-07: Released mxODBC Zope DA 2.1.2 ... http://egenix.com/go46 2013-05-06: Released mxODBC 3.2.3 ... http://egenix.com/go45 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From lord.mauve at gmail.com Sat May 18 13:24:47 2013 From: lord.mauve at gmail.com (Daniel Pope) Date: Sat, 18 May 2013 12:24:47 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: <51964118.4040402@egenix.com> References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> Message-ID: I thoroughly disagree with those who are saying that for small installations, it's less time-consuming to do things manually. A deployment/provisioning system gives you reproducibility - an executable record of how to re-create a server configuration or re-run a deployment without missing anything. The number of times I've spent 20 minutes wondering what I've missed all add up - that is why it automation breaks even so rapidly. Of course, you (or your employers) might have other reasons to want automation: - so that your deployment/provisioning is subject to automated testing, version control, code review, etc - so that you can build replicas of production servers for development, testing or disaster recovery - so that you can smash and rebuild a compromised or faulty machine without wasting time - so that you can deploy a dozen times a day and get features or fixes into the hands of your users On 17 May 2013 15:39, M.-A. Lemburg wrote: > On 16.05.2013 17:46, Andy Robinson wrote: > > Speaking as a relatively obsolete dinosaur, I would suggest that if > > you are going to discuss specific deployment practices, you start with > > the most fundamental ones: SSH, the unix shell and so on. > > > > We have had issues over the years with people coming in and > > introducing sexy new deployment tools, but ultimately they all just > > run unix commands. Anyone managing a web application in the > > non-microsoft world is ultimately depending on this. Some key skills > > (assuming a Linux/Mac/Unix-ish environment): > > - know about SSH keys and logging into remote machines > > - know the basics of at least one command line editor (e.g. vi) > > - basic shell knowledge: environment variables, testing for existence > > of files and directories etc > > - how to interact with your database from the command line, if you use > > one (including dump and restore) > > - how your web server works: starting, stopping, configuration files, > > where log files live and how it talks to Python > > > > Fabric may be useful if you want to control half a dozen machines from > > your desktop, and it might add a lot of value if you want to control a > > hundred of them. But to update one server, you deploy by logging into > > it and then running commands or short scripts. > > > > For example, we have a 'demo site' we rebuild pretty often that uses > > Django, MYSQL, Celery and a few other things. It runs on plain > > vanilla Ubuntu machines we build ourselves. The sequence is... > > > > 1. Log in via SSH > > 2. CD to correct directory > > 3. activate virtual environment > > 4. stop any celery worker processes > > 5. stop web server processes (* in our setup, we leave Apache running) > > 6. pull latest code from mercurial - both the app, and 3-4 libraries > > it depends on > > 7. run a management command to rebuild the database > > 8. run a smallish in-place test suite > > 9. restart celery workers > > 10.restart web server > > 11. log out > > > > All of this after the login and CD can be handled by a shell script on > > the path of the server, so you can just run a command called something > > like > > ./update_server > > > > More realistically, we tend to end up with a management shell script > > called 'server' with a bunch of commands/arguments like 'stop / start > > / restart / update-code-in-staging / copy-live-data-to-staging / > > run-health-checks / swap-live-and-staging' and so on. SSH can execute > > remote commands like this just fine with the right arguments, if > > actually logging in is too tedious. > > > > Production sites are complex and all different. You might want to do > > instantaneous swaps from live to staging (and be able to back out fast > > if stuff goes wrong); to switch DNS so the world is looking at another > > server while you update one; you might have large databases to copy or > > migrate that need significant time; it may or may not be acceptable to > > lose sessions and have downtime; and so on. > > > > > > It takes less time to learn the fundamentals than you will spend > > debugging why your fancy new deployment tool stopped working after > > some Python dependency upgrade somewhere. And it is less likely that > > your new hires will disagree if you stick with the lowest common > > denominator. > > Fully agreed. > > The new devops tools are nice when it comes to managing farms > of VMs, where each VM is setup in more or less the same way, > and you want to reduce repetitive tasks, but for a typical > setup with just a few VMs/servers it'll take you longer to > write and (most importantly) test your devops scripts than > it would to write a few scripts that you can run via SSH on > the servers. > > No matter how smart you make your devops scripts, Murphy's law > is inevitably going to strike and humans are so much better at > parsing weird intermittent error messages than machines ... > still, after all these years :-) > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, May 17 2013) > >>> Python Projects, Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2013-05-07: Released mxODBC Zope DA 2.1.2 ... http://egenix.com/go46 > 2013-05-06: Released mxODBC 3.2.3 ... http://egenix.com/go45 > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jjl at pobox.com Sat May 18 13:36:29 2013 From: jjl at pobox.com (John Lee) Date: Sat, 18 May 2013 12:36:29 +0100 (BST) Subject: [python-uk] CI and deployment (was Re: Suggestions / best practices for deployment) In-Reply-To: References: <51936F5A.1040901@tangentlabs.co.uk> Message-ID: Speaking of CI and deployment -- Does anybody know of an open source implementation of the idea described in this blog post? Maybe somebody has one and could open-source it? http://googletesting.blogspot.co.uk/2011/06/testing-at-speed-and-scale-of-google.html We're not all google, but overkill nicely implemented is sometimes just convenient and powerful. In fact, contrary to what you might expect, I think some environments where development is smaller scale need this more than some "big product" development does: for example, companies that might do web app development for multiple clients, where you have a fair amount of code you reuse and multiple pieces of work going on at the same time. In that situation -- unlike a big monolithic product -- you might want to keep some flexibility about which versions of code projects work together so that old apps can easily take advantage of new code when needed. This seems like a good way to test that your dependency declarations correspond to reality, and of course to detect that a change to code project A has broken code project B earlier rather than later (even though production is pinned to old releases, you probably want to find out about the breakage early because it might get picked up later on in production). VMs and deployment remind me of this because I think I recall a certain ex-colleague, as part of a scheme like this, wanting to build VMs to check whether system build inputs have changed. Something like that... Also, one "overkill" use I have in mind involves testing against fakes, when your application depends on some "heavy" systems that are slow and flaky to build and run -- probably built as a VM or a set of VMS. The tests for the heavy system would depend on both the real system and the fake, and tests of applications would depend on the fake. When the real system changes, the VMs get rebuilt and both the fake and the real thing automatically get retested. If any of those tests fail and somebody fixes the fake to match reality, the application tests get automatically re-run against the fixed fake. John On Wed, 15 May 2013, Stestagg wrote: > To add a slightly different angle to this, whatever deployment solution you > use, make sure it is fully automated, and then hook it into you CI system. > > Deployment, disaster recovery etc. are made so much simpler if you're > thinking about it from the start rather than just before release. One way > to enforce this is to start running the process regularly. > > My personal preference for this is to use snapshotted virtualbox VMs to > deploy the service/app to a clean OS install, and the run the tests, after > every commit (or on a schedule). This way, you always know what > dependencies you need, and you're forced to make things as seamless as > possible. > > The same goes for testing upgrades. Set up a VM with version X and set up > a job to: Upgrade > Test > Check > Rollback regularly. > > Thanks > > Steve > > > > > On Wed, May 15, 2013 at 12:19 PM, Muhammad Rahman < > muhammad.rahman at tangentlabs.co.uk> wrote: > >> Hi Harry, >> >> I would use salt(http://docs.saltstack.com/index.html) for provisioning >> and keep the python specific package to fabric to deploy using pip. >> >> Even though salt is able to deploy pip requirements but I think this >> separation is important. >> >> Mustafiz. >> >> >> On 15/05/2013 11:34, George Hickman wrote: >> >> Hi Harry, >> >> I use two methods for deployment - Heroku and Ubuntu VPSs. >> >> Heroku is really simple to get going but gets expensive if you want to >> run a serious production app. However it's probably what I'd recommend for >> beginners since so much is done for you and they have first class python >> support. >> >> For VPS deploys I use Nginx, Daemontools (for the envdir package), >> Supervisor, Gunicorn and Postgres (with Django). I usually deploy this >> setup with a mixture of Salt to build the VPS and Fabric to deploy the code. >> >> George >> >> >> On Wed, May 15, 2013 at 10:57 AM, Harry Percival >> wrote: >> >>> Dear UK Python chums, >>> >>> some of you probably know I'm writing a book about TDD for O'Reilly. >>> I'm looking for some help with the (first) chapter on deployment. >>> >>> http://www.obeythetestinggoat.com/what-to-say-about-deployment.html >>> >>> What do you use for deployment? Do you have any kind of automated >>> scripts? How do you manage virtualenvs, the database, apache/uwsgi >>> config... What do you think might work as a sort of "best practice lite" >>> for a simple site for beginners? (django, sqlite database, static files) >>> >>> -- >>> ------------------------------ >>> Harry J.W. Percival >>> ------------------------------ >>> Twitter: @hjwp >>> Mobile: +44 (0) 78877 02511 <%2B44%20%280%29%2078877%2002511> >>> Skype: harry.percival >>> >>> _______________________________________________ >>> python-uk mailing list >>> python-uk at python.org >>> http://mail.python.org/mailman/listinfo/python-uk >>> >>> >> >> >> _______________________________________________ >> python-uk mailing listpython-uk at python.orghttp://mail.python.org/mailman/listinfo/python-uk >> >> >> -- >> >> Muhammad Rahman >> Developer >> >> Tangent Labs >> 84 - 86 Great Portland Street >> London, W1W 7NR >> >> T: +44 (0)207 462 6150 (Office) >> F: +44 (0)207 462 6111 >> W: www.tangentlabs.co.uk >> >> >> _______________________________________________ >> python-uk mailing list >> python-uk at python.org >> http://mail.python.org/mailman/listinfo/python-uk >> >> > From a.cavallo at cavallinux.eu Sat May 18 16:27:24 2013 From: a.cavallo at cavallinux.eu (Antonio Cavallo) Date: Sat, 18 May 2013 15:27:24 +0100 Subject: [python-uk] CI and deployment (was Re: Suggestions / best practices for deployment) In-Reply-To: References: <51936F5A.1040901@tangentlabs.co.uk> Message-ID: <4AED6FAB-15BE-456B-97FA-A43002EEE2ED@cavallinux.eu> I don't know any scenario like that: usually dependencies are frozen (and tested) separately in a "release". It makes the whole process simpler. Possibly they wanted to test how far they can push TDD, to be able to do testing on the entire code (apps + libraries) using the development branch (all commits done on it), api changes in the clients breaking the servers? I hope you won't find *any* CI doing that: that'd be very "code specific" nothing that a CI should be aware of :D I hope this helps On 18 May 2013, at 12:36, John Lee wrote: > Speaking of CI and deployment -- Does anybody know of an open source implementation of the idea described in this blog post? Maybe somebody has one and could open-source it? > > http://googletesting.blogspot.co.uk/2011/06/testing-at-speed-and-scale-of-google.html > > > We're not all google, but overkill nicely implemented is sometimes just convenient and powerful. From andy at reportlab.com Sat May 18 18:41:14 2013 From: andy at reportlab.com (Andy Robinson) Date: Sat, 18 May 2013 17:41:14 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> Message-ID: Daniel, who are you disagreeing with? Everyone here agrees on automation, I think. - Andy On 18 May 2013 12:24, Daniel Pope wrote: > I thoroughly disagree with those who are saying that for small > installations, it's less time-consuming to do things manually. A > deployment/provisioning system gives you reproducibility - an executable > record of how to re-create a server configuration or re-run a deployment > without missing anything. The number of times I've spent 20 minutes > wondering what I've missed all add up - that is why it automation breaks > even so rapidly. > > Of course, you (or your employers) might have other reasons to want > automation: > > - so that your deployment/provisioning is subject to automated testing, > version control, code review, etc > - so that you can build replicas of production servers for development, > testing or disaster recovery > - so that you can smash and rebuild a compromised or faulty machine without > wasting time > - so that you can deploy a dozen times a day and get features or fixes into > the hands of your users > > > On 17 May 2013 15:39, M.-A. Lemburg wrote: >> >> On 16.05.2013 17:46, Andy Robinson wrote: >> > Speaking as a relatively obsolete dinosaur, I would suggest that if >> > you are going to discuss specific deployment practices, you start with >> > the most fundamental ones: SSH, the unix shell and so on. >> > >> > We have had issues over the years with people coming in and >> > introducing sexy new deployment tools, but ultimately they all just >> > run unix commands. Anyone managing a web application in the >> > non-microsoft world is ultimately depending on this. Some key skills >> > (assuming a Linux/Mac/Unix-ish environment): >> > - know about SSH keys and logging into remote machines >> > - know the basics of at least one command line editor (e.g. vi) >> > - basic shell knowledge: environment variables, testing for existence >> > of files and directories etc >> > - how to interact with your database from the command line, if you use >> > one (including dump and restore) >> > - how your web server works: starting, stopping, configuration files, >> > where log files live and how it talks to Python >> > >> > Fabric may be useful if you want to control half a dozen machines from >> > your desktop, and it might add a lot of value if you want to control a >> > hundred of them. But to update one server, you deploy by logging into >> > it and then running commands or short scripts. >> > >> > For example, we have a 'demo site' we rebuild pretty often that uses >> > Django, MYSQL, Celery and a few other things. It runs on plain >> > vanilla Ubuntu machines we build ourselves. The sequence is... >> > >> > 1. Log in via SSH >> > 2. CD to correct directory >> > 3. activate virtual environment >> > 4. stop any celery worker processes >> > 5. stop web server processes (* in our setup, we leave Apache running) >> > 6. pull latest code from mercurial - both the app, and 3-4 libraries >> > it depends on >> > 7. run a management command to rebuild the database >> > 8. run a smallish in-place test suite >> > 9. restart celery workers >> > 10.restart web server >> > 11. log out >> > >> > All of this after the login and CD can be handled by a shell script on >> > the path of the server, so you can just run a command called something >> > like >> > ./update_server >> > >> > More realistically, we tend to end up with a management shell script >> > called 'server' with a bunch of commands/arguments like 'stop / start >> > / restart / update-code-in-staging / copy-live-data-to-staging / >> > run-health-checks / swap-live-and-staging' and so on. SSH can execute >> > remote commands like this just fine with the right arguments, if >> > actually logging in is too tedious. >> > >> > Production sites are complex and all different. You might want to do >> > instantaneous swaps from live to staging (and be able to back out fast >> > if stuff goes wrong); to switch DNS so the world is looking at another >> > server while you update one; you might have large databases to copy or >> > migrate that need significant time; it may or may not be acceptable to >> > lose sessions and have downtime; and so on. >> > >> > >> > It takes less time to learn the fundamentals than you will spend >> > debugging why your fancy new deployment tool stopped working after >> > some Python dependency upgrade somewhere. And it is less likely that >> > your new hires will disagree if you stick with the lowest common >> > denominator. >> >> Fully agreed. >> >> The new devops tools are nice when it comes to managing farms >> of VMs, where each VM is setup in more or less the same way, >> and you want to reduce repetitive tasks, but for a typical >> setup with just a few VMs/servers it'll take you longer to >> write and (most importantly) test your devops scripts than >> it would to write a few scripts that you can run via SSH on >> the servers. >> >> No matter how smart you make your devops scripts, Murphy's law >> is inevitably going to strike and humans are so much better at >> parsing weird intermittent error messages than machines ... >> still, after all these years :-) >> >> -- >> Marc-Andre Lemburg >> eGenix.com >> >> Professional Python Services directly from the Source (#1, May 17 2013) >> >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >> >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >> >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ >> ________________________________________________________________________ >> 2013-05-07: Released mxODBC Zope DA 2.1.2 ... http://egenix.com/go46 >> 2013-05-06: Released mxODBC 3.2.3 ... http://egenix.com/go45 >> >> ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: >> >> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 >> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg >> Registered at Amtsgericht Duesseldorf: HRB 46611 >> http://www.egenix.com/company/contact/ >> _______________________________________________ >> python-uk mailing list >> python-uk at python.org >> http://mail.python.org/mailman/listinfo/python-uk > > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > -- Andy Robinson Managing Director ReportLab Europe Ltd. Thornton House, Thornton Road, Wimbledon, London SW19 4NG, UK Tel +44-20-8405-6420 From kris at automationlogic.com Sat May 18 19:10:13 2013 From: kris at automationlogic.com (Kris Saxton) Date: Sat, 18 May 2013 18:10:13 +0100 Subject: [python-uk] Suggestions / best practices for deployment Message-ID: <054FB809-95D2-402C-B54B-1A5086519C1E@automationlogic.com> Just to second what Daniel is saying, I have found that generally we get much higher quality outcomes using a deployment/management system (puppet/chef/salt) over SSH/shell commands, such that I would even use it to manage repetitive tasks on a single machine. Kris -- Kris Saxton e: kris at automationlogic.com t: @KrisSaxton From mike at mikedeplume.com Mon May 20 12:17:33 2013 From: mike at mikedeplume.com (MikedePlume) Date: Mon, 20 May 2013 11:17:33 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> Message-ID: <1369045053.2123.28.camel@oberto> On Sat, 2013-05-18 at 17:41 +0100, Andy Robinson wrote: > Daniel, who are you disagreeing with? Everyone here agrees on > automation, I think. > > - Andy As a small user (one or two servers, one or two packages to deploy): I started by doing a lot of typing of commands, got bored with that, moved on to command line scripts and I now use a bunch of (python) scripts, some fabric based, to manage the server farm api and the ssh stuff. I think the point is that the underlying process is always going to involve ssh and some server farm api and it is the automation of all of that where the cornucopia of choice (?) is displayed. >From a teaching perspective (that's what the book is about, I guess) I'd recognise the basics of ssh and so on and then build up from there. The basic principles behind testing a fabric deploy should be extendable to other environments, I'd have thought? I must say, I'd love to see something about testing bash scripts, and, perhaps, where to draw the line (if at all) between server provisioning and ci. Cheers Mike S. > > > > On 18 May 2013 12:24, Daniel Pope wrote: > > I thoroughly disagree with those who are saying that for small > > installations, it's less time-consuming to do things manually. A > > deployment/provisioning system gives you reproducibility - an executable > > record of how to re-create a server configuration or re-run a deployment > > without missing anything. The number of times I've spent 20 minutes > > wondering what I've missed all add up - that is why it automation breaks > > even so rapidly. > > > > Of course, you (or your employers) might have other reasons to want > > automation: > > > > - so that your deployment/provisioning is subject to automated testing, > > version control, code review, etc > > - so that you can build replicas of production servers for development, > > testing or disaster recovery > > - so that you can smash and rebuild a compromised or faulty machine without > > wasting time > > - so that you can deploy a dozen times a day and get features or fixes into > > the hands of your users > > > > > > On 17 May 2013 15:39, M.-A. Lemburg wrote: > >> > >> On 16.05.2013 17:46, Andy Robinson wrote: > >> > Speaking as a relatively obsolete dinosaur, I would suggest that if > >> > you are going to discuss specific deployment practices, you start with > >> > the most fundamental ones: SSH, the unix shell and so on. > >> > > >> > We have had issues over the years with people coming in and > >> > introducing sexy new deployment tools, but ultimately they all just > >> > run unix commands. Anyone managing a web application in the > >> > non-microsoft world is ultimately depending on this. Some key skills > >> > (assuming a Linux/Mac/Unix-ish environment): > >> > - know about SSH keys and logging into remote machines > >> > - know the basics of at least one command line editor (e.g. vi) > >> > - basic shell knowledge: environment variables, testing for existence > >> > of files and directories etc > >> > - how to interact with your database from the command line, if you use > >> > one (including dump and restore) > >> > - how your web server works: starting, stopping, configuration files, > >> > where log files live and how it talks to Python > >> > > >> > Fabric may be useful if you want to control half a dozen machines from > >> > your desktop, and it might add a lot of value if you want to control a > >> > hundred of them. But to update one server, you deploy by logging into > >> > it and then running commands or short scripts. > >> > > >> > For example, we have a 'demo site' we rebuild pretty often that uses > >> > Django, MYSQL, Celery and a few other things. It runs on plain > >> > vanilla Ubuntu machines we build ourselves. The sequence is... > >> > > >> > 1. Log in via SSH > >> > 2. CD to correct directory > >> > 3. activate virtual environment > >> > 4. stop any celery worker processes > >> > 5. stop web server processes (* in our setup, we leave Apache running) > >> > 6. pull latest code from mercurial - both the app, and 3-4 libraries > >> > it depends on > >> > 7. run a management command to rebuild the database > >> > 8. run a smallish in-place test suite > >> > 9. restart celery workers > >> > 10.restart web server > >> > 11. log out > >> > > >> > All of this after the login and CD can be handled by a shell script on > >> > the path of the server, so you can just run a command called something > >> > like > >> > ./update_server > >> > > >> > More realistically, we tend to end up with a management shell script > >> > called 'server' with a bunch of commands/arguments like 'stop / start > >> > / restart / update-code-in-staging / copy-live-data-to-staging / > >> > run-health-checks / swap-live-and-staging' and so on. SSH can execute > >> > remote commands like this just fine with the right arguments, if > >> > actually logging in is too tedious. > >> > > >> > Production sites are complex and all different. You might want to do > >> > instantaneous swaps from live to staging (and be able to back out fast > >> > if stuff goes wrong); to switch DNS so the world is looking at another > >> > server while you update one; you might have large databases to copy or > >> > migrate that need significant time; it may or may not be acceptable to > >> > lose sessions and have downtime; and so on. > >> > > >> > > >> > It takes less time to learn the fundamentals than you will spend > >> > debugging why your fancy new deployment tool stopped working after > >> > some Python dependency upgrade somewhere. And it is less likely that > >> > your new hires will disagree if you stick with the lowest common > >> > denominator. > >> > >> Fully agreed. > >> > >> The new devops tools are nice when it comes to managing farms > >> of VMs, where each VM is setup in more or less the same way, > >> and you want to reduce repetitive tasks, but for a typical > >> setup with just a few VMs/servers it'll take you longer to > >> write and (most importantly) test your devops scripts than > >> it would to write a few scripts that you can run via SSH on > >> the servers. > >> > >> No matter how smart you make your devops scripts, Murphy's law > >> is inevitably going to strike and humans are so much better at > >> parsing weird intermittent error messages than machines ... > >> still, after all these years :-) > >> > >> -- > >> Marc-Andre Lemburg > >> eGenix.com > >> > >> Professional Python Services directly from the Source (#1, May 17 2013) > >> >>> Python Projects, Consulting and Support ... http://www.egenix.com/ > >> >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ > >> >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > >> ________________________________________________________________________ > >> 2013-05-07: Released mxODBC Zope DA 2.1.2 ... http://egenix.com/go46 > >> 2013-05-06: Released mxODBC 3.2.3 ... http://egenix.com/go45 > >> > >> ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > >> > >> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > >> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > >> Registered at Amtsgericht Duesseldorf: HRB 46611 > >> http://www.egenix.com/company/contact/ > >> _______________________________________________ > >> python-uk mailing list > >> python-uk at python.org > >> http://mail.python.org/mailman/listinfo/python-uk > > > > > > > > _______________________________________________ > > python-uk mailing list > > python-uk at python.org > > http://mail.python.org/mailman/listinfo/python-uk > > > > > From lord.mauve at gmail.com Mon May 20 14:03:33 2013 From: lord.mauve at gmail.com (Daniel Pope) Date: Mon, 20 May 2013 13:03:33 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: <1369045053.2123.28.camel@oberto> References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> Message-ID: It doesn't need to be SSH-based, there are orchestration systems that are RPC or pubsub based (eg. Jenkins, MCollective). As per the line between provisioning and deployment I've argued in the past that provisioning should be for state that spans many minor releases of an application. I also believe it's preferable to require that deployment cannot use root permissions, as this is less likely to impact independent services hosted on the same machine. It also allows better auditability - it prevents deployments making changes that are rightly the domain of the provisioning system. On 20 May 2013 11:17, "MikedePlume" wrote: > On Sat, 2013-05-18 at 17:41 +0100, Andy Robinson wrote: > > Daniel, who are you disagreeing with? Everyone here agrees on > > automation, I think. > > > > - Andy > > As a small user (one or two servers, one or two packages to deploy): > > I started by doing a lot of typing of commands, got bored with that, > moved on to command line scripts and I now use a bunch of (python) > scripts, some fabric based, to manage the server farm api and the ssh > stuff. I think the point is that the underlying process is always going > to involve ssh and some server farm api and it is the automation of all > of that where the cornucopia of choice (?) is displayed. > > From a teaching perspective (that's what the book is about, I guess) I'd > recognise the basics of ssh and so on and then build up from there. The > basic principles behind testing a fabric deploy should be extendable to > other environments, I'd have thought? > > I must say, I'd love to see something about testing bash scripts, and, > perhaps, where to draw the line (if at all) between server provisioning > and ci. > > Cheers > > Mike S. > > > > > > > > On 18 May 2013 12:24, Daniel Pope wrote: > > > I thoroughly disagree with those who are saying that for small > > > installations, it's less time-consuming to do things manually. A > > > deployment/provisioning system gives you reproducibility - an > executable > > > record of how to re-create a server configuration or re-run a > deployment > > > without missing anything. The number of times I've spent 20 minutes > > > wondering what I've missed all add up - that is why it automation > breaks > > > even so rapidly. > > > > > > Of course, you (or your employers) might have other reasons to want > > > automation: > > > > > > - so that your deployment/provisioning is subject to automated testing, > > > version control, code review, etc > > > - so that you can build replicas of production servers for development, > > > testing or disaster recovery > > > - so that you can smash and rebuild a compromised or faulty machine > without > > > wasting time > > > - so that you can deploy a dozen times a day and get features or fixes > into > > > the hands of your users > > > > > > > > > On 17 May 2013 15:39, M.-A. Lemburg wrote: > > >> > > >> On 16.05.2013 17:46, Andy Robinson wrote: > > >> > Speaking as a relatively obsolete dinosaur, I would suggest that if > > >> > you are going to discuss specific deployment practices, you start > with > > >> > the most fundamental ones: SSH, the unix shell and so on. > > >> > > > >> > We have had issues over the years with people coming in and > > >> > introducing sexy new deployment tools, but ultimately they all just > > >> > run unix commands. Anyone managing a web application in the > > >> > non-microsoft world is ultimately depending on this. Some key > skills > > >> > (assuming a Linux/Mac/Unix-ish environment): > > >> > - know about SSH keys and logging into remote machines > > >> > - know the basics of at least one command line editor (e.g. vi) > > >> > - basic shell knowledge: environment variables, testing for > existence > > >> > of files and directories etc > > >> > - how to interact with your database from the command line, if you > use > > >> > one (including dump and restore) > > >> > - how your web server works: starting, stopping, configuration > files, > > >> > where log files live and how it talks to Python > > >> > > > >> > Fabric may be useful if you want to control half a dozen machines > from > > >> > your desktop, and it might add a lot of value if you want to > control a > > >> > hundred of them. But to update one server, you deploy by logging > into > > >> > it and then running commands or short scripts. > > >> > > > >> > For example, we have a 'demo site' we rebuild pretty often that uses > > >> > Django, MYSQL, Celery and a few other things. It runs on plain > > >> > vanilla Ubuntu machines we build ourselves. The sequence is... > > >> > > > >> > 1. Log in via SSH > > >> > 2. CD to correct directory > > >> > 3. activate virtual environment > > >> > 4. stop any celery worker processes > > >> > 5. stop web server processes (* in our setup, we leave Apache > running) > > >> > 6. pull latest code from mercurial - both the app, and 3-4 libraries > > >> > it depends on > > >> > 7. run a management command to rebuild the database > > >> > 8. run a smallish in-place test suite > > >> > 9. restart celery workers > > >> > 10.restart web server > > >> > 11. log out > > >> > > > >> > All of this after the login and CD can be handled by a shell script > on > > >> > the path of the server, so you can just run a command called > something > > >> > like > > >> > ./update_server > > >> > > > >> > More realistically, we tend to end up with a management shell script > > >> > called 'server' with a bunch of commands/arguments like 'stop / > start > > >> > / restart / update-code-in-staging / copy-live-data-to-staging / > > >> > run-health-checks / swap-live-and-staging' and so on. SSH can > execute > > >> > remote commands like this just fine with the right arguments, if > > >> > actually logging in is too tedious. > > >> > > > >> > Production sites are complex and all different. You might want to > do > > >> > instantaneous swaps from live to staging (and be able to back out > fast > > >> > if stuff goes wrong); to switch DNS so the world is looking at > another > > >> > server while you update one; you might have large databases to copy > or > > >> > migrate that need significant time; it may or may not be acceptable > to > > >> > lose sessions and have downtime; and so on. > > >> > > > >> > > > >> > It takes less time to learn the fundamentals than you will spend > > >> > debugging why your fancy new deployment tool stopped working after > > >> > some Python dependency upgrade somewhere. And it is less likely > that > > >> > your new hires will disagree if you stick with the lowest common > > >> > denominator. > > >> > > >> Fully agreed. > > >> > > >> The new devops tools are nice when it comes to managing farms > > >> of VMs, where each VM is setup in more or less the same way, > > >> and you want to reduce repetitive tasks, but for a typical > > >> setup with just a few VMs/servers it'll take you longer to > > >> write and (most importantly) test your devops scripts than > > >> it would to write a few scripts that you can run via SSH on > > >> the servers. > > >> > > >> No matter how smart you make your devops scripts, Murphy's law > > >> is inevitably going to strike and humans are so much better at > > >> parsing weird intermittent error messages than machines ... > > >> still, after all these years :-) > > >> > > >> -- > > >> Marc-Andre Lemburg > > >> eGenix.com > > >> > > >> Professional Python Services directly from the Source (#1, May 17 > 2013) > > >> >>> Python Projects, Consulting and Support ... > http://www.egenix.com/ > > >> >>> mxODBC.Zope/Plone.Database.Adapter ... > http://zope.egenix.com/ > > >> >>> mxODBC, mxDateTime, mxTextTools ... > http://python.egenix.com/ > > >> > ________________________________________________________________________ > > >> 2013-05-07: Released mxODBC Zope DA 2.1.2 ... > http://egenix.com/go46 > > >> 2013-05-06: Released mxODBC 3.2.3 ... > http://egenix.com/go45 > > >> > > >> ::::: Try our mxODBC.Connect Python Database Interface for free ! > :::::: > > >> > > >> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > > >> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > > >> Registered at Amtsgericht Duesseldorf: HRB 46611 > > >> http://www.egenix.com/company/contact/ > > >> _______________________________________________ > > >> python-uk mailing list > > >> python-uk at python.org > > >> http://mail.python.org/mailman/listinfo/python-uk > > > > > > > > > > > > _______________________________________________ > > > python-uk mailing list > > > python-uk at python.org > > > http://mail.python.org/mailman/listinfo/python-uk > > > > > > > > > > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harry.percival at gmail.com Mon May 20 15:51:40 2013 From: harry.percival at gmail.com (Harry Percival) Date: Mon, 20 May 2013 14:51:40 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> Message-ID: Interesting points both. I think it's insightful to think about the line beween provisioning and deployment... The idea that deployment shouldn't require root permissions is a good stake in the ground. In the book I've decided I'm going to take people through spinning up a server with Apache, and configuring two virtualhosts with mod_wsgi, one for staging and one for production. We negotiate the gotchas, including setting up static files, using absolute paths and getting the permissions right for the (sqlite) database, etc. Then, deployment is a matter of updating the source code, updating any static files, and bouncing apache. Provisioning: - spinning up a server - installing apache - configuring apache Deployment: -updating source code -updating static files -migrate database -restart apache For those, the root/non-root perms thing holds pretty well, apart from the "restart apache" bit... And, up to a certain point, what I talk about under "provisioning" is stuff that might vary from platform to platform, but stuff under "deployment" should be more universal... For automating, I think they can be two separate scripts... Although we might keep "spinning up a server" manual... What about testing? Obviously both scripts will be tested indirectly via my functional tests (which I'm planning to run against staging, and they will show up any, eg, CSS problems)... But I've been pretty hot in the book on unit testing everything too. At PA, our deploy scripts aren't unit tested, and as a result they're a bit of a mess. But, unit testing fabric scripts? Won't it end up being a series of mocks? I find those kinds of tests a bit pointless... then again, maybe if we'd had mocky unit tests for all our deploy scripts since the beginning, perhaps they'd be better structured now... Thoughts? Also, question for django people: static files, as collected by "manage.py collectstatic": in the repo, or not? On 20 May 2013 13:03, Daniel Pope wrote: > It doesn't need to be SSH-based, there are orchestration systems that are > RPC or pubsub based (eg. Jenkins, MCollective). > > As per the line between provisioning and deployment I've argued in the > past that provisioning should be for state that spans many minor releases > of an application. I also believe it's preferable to require that > deployment cannot use root permissions, as this is less likely to impact > independent services hosted on the same machine. It also allows better > auditability - it prevents deployments making changes that are rightly the > domain of the provisioning system. > On 20 May 2013 11:17, "MikedePlume" wrote: > >> On Sat, 2013-05-18 at 17:41 +0100, Andy Robinson wrote: >> > Daniel, who are you disagreeing with? Everyone here agrees on >> > automation, I think. >> > >> > - Andy >> >> As a small user (one or two servers, one or two packages to deploy): >> >> I started by doing a lot of typing of commands, got bored with that, >> moved on to command line scripts and I now use a bunch of (python) >> scripts, some fabric based, to manage the server farm api and the ssh >> stuff. I think the point is that the underlying process is always going >> to involve ssh and some server farm api and it is the automation of all >> of that where the cornucopia of choice (?) is displayed. >> >> From a teaching perspective (that's what the book is about, I guess) I'd >> recognise the basics of ssh and so on and then build up from there. The >> basic principles behind testing a fabric deploy should be extendable to >> other environments, I'd have thought? >> >> I must say, I'd love to see something about testing bash scripts, and, >> perhaps, where to draw the line (if at all) between server provisioning >> and ci. >> >> Cheers >> >> Mike S. >> > >> > >> > >> > On 18 May 2013 12:24, Daniel Pope wrote: >> > > I thoroughly disagree with those who are saying that for small >> > > installations, it's less time-consuming to do things manually. A >> > > deployment/provisioning system gives you reproducibility - an >> executable >> > > record of how to re-create a server configuration or re-run a >> deployment >> > > without missing anything. The number of times I've spent 20 minutes >> > > wondering what I've missed all add up - that is why it automation >> breaks >> > > even so rapidly. >> > > >> > > Of course, you (or your employers) might have other reasons to want >> > > automation: >> > > >> > > - so that your deployment/provisioning is subject to automated >> testing, >> > > version control, code review, etc >> > > - so that you can build replicas of production servers for >> development, >> > > testing or disaster recovery >> > > - so that you can smash and rebuild a compromised or faulty machine >> without >> > > wasting time >> > > - so that you can deploy a dozen times a day and get features or >> fixes into >> > > the hands of your users >> > > >> > > >> > > On 17 May 2013 15:39, M.-A. Lemburg wrote: >> > >> >> > >> On 16.05.2013 17:46, Andy Robinson wrote: >> > >> > Speaking as a relatively obsolete dinosaur, I would suggest that if >> > >> > you are going to discuss specific deployment practices, you start >> with >> > >> > the most fundamental ones: SSH, the unix shell and so on. >> > >> > >> > >> > We have had issues over the years with people coming in and >> > >> > introducing sexy new deployment tools, but ultimately they all just >> > >> > run unix commands. Anyone managing a web application in the >> > >> > non-microsoft world is ultimately depending on this. Some key >> skills >> > >> > (assuming a Linux/Mac/Unix-ish environment): >> > >> > - know about SSH keys and logging into remote machines >> > >> > - know the basics of at least one command line editor (e.g. vi) >> > >> > - basic shell knowledge: environment variables, testing for >> existence >> > >> > of files and directories etc >> > >> > - how to interact with your database from the command line, if you >> use >> > >> > one (including dump and restore) >> > >> > - how your web server works: starting, stopping, configuration >> files, >> > >> > where log files live and how it talks to Python >> > >> > >> > >> > Fabric may be useful if you want to control half a dozen machines >> from >> > >> > your desktop, and it might add a lot of value if you want to >> control a >> > >> > hundred of them. But to update one server, you deploy by logging >> into >> > >> > it and then running commands or short scripts. >> > >> > >> > >> > For example, we have a 'demo site' we rebuild pretty often that >> uses >> > >> > Django, MYSQL, Celery and a few other things. It runs on plain >> > >> > vanilla Ubuntu machines we build ourselves. The sequence is... >> > >> > >> > >> > 1. Log in via SSH >> > >> > 2. CD to correct directory >> > >> > 3. activate virtual environment >> > >> > 4. stop any celery worker processes >> > >> > 5. stop web server processes (* in our setup, we leave Apache >> running) >> > >> > 6. pull latest code from mercurial - both the app, and 3-4 >> libraries >> > >> > it depends on >> > >> > 7. run a management command to rebuild the database >> > >> > 8. run a smallish in-place test suite >> > >> > 9. restart celery workers >> > >> > 10.restart web server >> > >> > 11. log out >> > >> > >> > >> > All of this after the login and CD can be handled by a shell >> script on >> > >> > the path of the server, so you can just run a command called >> something >> > >> > like >> > >> > ./update_server >> > >> > >> > >> > More realistically, we tend to end up with a management shell >> script >> > >> > called 'server' with a bunch of commands/arguments like 'stop / >> start >> > >> > / restart / update-code-in-staging / copy-live-data-to-staging / >> > >> > run-health-checks / swap-live-and-staging' and so on. SSH can >> execute >> > >> > remote commands like this just fine with the right arguments, if >> > >> > actually logging in is too tedious. >> > >> > >> > >> > Production sites are complex and all different. You might want to >> do >> > >> > instantaneous swaps from live to staging (and be able to back out >> fast >> > >> > if stuff goes wrong); to switch DNS so the world is looking at >> another >> > >> > server while you update one; you might have large databases to >> copy or >> > >> > migrate that need significant time; it may or may not be >> acceptable to >> > >> > lose sessions and have downtime; and so on. >> > >> > >> > >> > >> > >> > It takes less time to learn the fundamentals than you will spend >> > >> > debugging why your fancy new deployment tool stopped working after >> > >> > some Python dependency upgrade somewhere. And it is less likely >> that >> > >> > your new hires will disagree if you stick with the lowest common >> > >> > denominator. >> > >> >> > >> Fully agreed. >> > >> >> > >> The new devops tools are nice when it comes to managing farms >> > >> of VMs, where each VM is setup in more or less the same way, >> > >> and you want to reduce repetitive tasks, but for a typical >> > >> setup with just a few VMs/servers it'll take you longer to >> > >> write and (most importantly) test your devops scripts than >> > >> it would to write a few scripts that you can run via SSH on >> > >> the servers. >> > >> >> > >> No matter how smart you make your devops scripts, Murphy's law >> > >> is inevitably going to strike and humans are so much better at >> > >> parsing weird intermittent error messages than machines ... >> > >> still, after all these years :-) >> > >> >> > >> -- >> > >> Marc-Andre Lemburg >> > >> eGenix.com >> > >> >> > >> Professional Python Services directly from the Source (#1, May 17 >> 2013) >> > >> >>> Python Projects, Consulting and Support ... >> http://www.egenix.com/ >> > >> >>> mxODBC.Zope/Plone.Database.Adapter ... >> http://zope.egenix.com/ >> > >> >>> mxODBC, mxDateTime, mxTextTools ... >> http://python.egenix.com/ >> > >> >> ________________________________________________________________________ >> > >> 2013-05-07: Released mxODBC Zope DA 2.1.2 ... >> http://egenix.com/go46 >> > >> 2013-05-06: Released mxODBC 3.2.3 ... >> http://egenix.com/go45 >> > >> >> > >> ::::: Try our mxODBC.Connect Python Database Interface for free ! >> :::::: >> > >> >> > >> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 >> > >> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg >> > >> Registered at Amtsgericht Duesseldorf: HRB 46611 >> > >> http://www.egenix.com/company/contact/ >> > >> _______________________________________________ >> > >> python-uk mailing list >> > >> python-uk at python.org >> > >> http://mail.python.org/mailman/listinfo/python-uk >> > > >> > > >> > > >> > > _______________________________________________ >> > > python-uk mailing list >> > > python-uk at python.org >> > > http://mail.python.org/mailman/listinfo/python-uk >> > > >> > >> > >> > >> >> >> _______________________________________________ >> python-uk mailing list >> python-uk at python.org >> http://mail.python.org/mailman/listinfo/python-uk >> > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > -- ------------------------------ Harry J.W. Percival ------------------------------ Twitter: @hjwp Mobile: +44 (0) 78877 02511 Skype: harry.percival -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at reynoldsfamily.org.uk Mon May 20 16:00:13 2013 From: david at reynoldsfamily.org.uk (David Reynolds) Date: Mon, 20 May 2013 15:00:13 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> Message-ID: <18C4019E-5E9F-4219-A1A1-95A51CFFACF5@reynoldsfamily.org.uk> On 20 May 2013, at 14:51, Harry Percival wrote: > Also, question for django people: static files, as collected by "manage.py collectstatic": in the repo, or not? Not. That is a deployment step in my book. -- David Reynolds david at reynoldsfamily.org.uk From fuzzyman at voidspace.org.uk Mon May 20 16:22:56 2013 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 20 May 2013 15:22:56 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: <18C4019E-5E9F-4219-A1A1-95A51CFFACF5@reynoldsfamily.org.uk> References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> <18C4019E-5E9F-4219-A1A1-95A51CFFACF5@reynoldsfamily.org.uk> Message-ID: <3A170204-0DE4-4AF5-8ED7-A039F28D317A@voidspace.org.uk> On 20 May 2013, at 15:00, David Reynolds wrote: > > On 20 May 2013, at 14:51, Harry Percival wrote: > >> Also, question for django people: static files, as collected by "manage.py collectstatic": in the repo, or not? > > Not. That is a deployment step in my book. > > Agreed. Many apps will copy their static files from the installed version when you issue this command and there's just no need for those to be in the repo. Michael > -- > David Reynolds > david at reynoldsfamily.org.uk > > > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html From tim at red56.co.uk Mon May 20 16:38:52 2013 From: tim at red56.co.uk (Tim Diggins) Date: Mon, 20 May 2013 15:38:52 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> Message-ID: On 20 May 2013 14:51, Harry Percival wrote: > Interesting points both. I think it's insightful to think about the line > beween provisioning and deployment... The idea that deployment shouldn't > require root permissions is a good stake in the ground. > > In the book I've decided I'm going to take people through spinning up a > server with Apache, and configuring two virtualhosts with mod_wsgi, one for > staging and one for production. We negotiate the gotchas, including > setting up static files, using absolute paths and getting the permissions > right for the (sqlite) database, etc. > > Then, deployment is a matter of updating the source code, updating any > static files, and bouncing apache. > > Provisioning: > - spinning up a server > - installing apache > - configuring apache > > Deployment: > -updating source code > -updating static files > -migrate database > -restart apache > > For those, the root/non-root perms thing holds pretty well, apart from the > "restart apache" bit... And, up to a certain point, what I talk about > under "provisioning" is stuff that might vary from platform to platform, > but stuff under "deployment" should be more universal... > > you shouldn't need to restart apache/nginx. Touch the wsgi file and that part should restart automatically. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike at mikedeplume.com Mon May 20 16:40:28 2013 From: mike at mikedeplume.com (MikedePlume) Date: Mon, 20 May 2013 15:40:28 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> Message-ID: <1369060828.2123.67.camel@oberto> On Mon, 2013-05-20 at 14:51 +0100, Harry Percival wrote: > Interesting points both. I think it's insightful to think about the > line beween provisioning and deployment... The idea that deployment > shouldn't require root permissions is a good stake in the ground. > > > In the book I've decided I'm going to take people through spinning up > a server with Apache, and configuring two virtualhosts with mod_wsgi, > one for staging and one for production. We negotiate the gotchas, > including setting up static files, using absolute paths and getting > the permissions right for the (sqlite) database, etc. > > > Then, deployment is a matter of updating the source code, updating any > static files, and bouncing apache. > > > Provisioning: > - spinning up a server > - installing apache > - configuring apache > > Deployment: > -updating source code > -updating static files > -migrate database > -restart apache > For those, the root/non-root perms thing holds pretty well, apart from > the "restart apache" bit... And, up to a certain point, what I talk > about under "provisioning" is stuff that might vary from platform to > platform, but stuff under "deployment" should be more universal... Interesting that the subject veers towards system design. I use FCGI myself and the restart can be entirely non-root. Provisioning then includes adding the FCGI reference into the server config. > For automating, I think they can be two separate scripts... Although > we might keep "spinning up a server" manual... > > What about testing? Obviously both scripts will be tested indirectly > via my functional tests (which I'm planning to run against staging, > and they will show up any, eg, CSS problems)... But I've been pretty > hot in the book on unit testing everything too. At PA, our deploy > scripts aren't unit tested, and as a result they're a bit of a mess. > Yup. That's where I'm coming from. But maybe there is a reasonable cut-off? If a deployment script does not have too many paths (which sounds reasonable) then one test (the just-do-it one) may be sufficient? > But, unit testing fabric scripts? Won't it end up being a series of > mocks? I find those kinds of tests a bit pointless... then again, > maybe if we'd had mocky unit tests for all our deploy scripts since > the beginning, perhaps they'd be better structured now... Thoughts? It _is_ possible to test deployment scripts (against a VM perhaps) to check that directories are created and so on, but I've tried to make sure that pretty much all of the files (config or other set-up, and static files) are part of the deployment itself and will be copies of whatever happened in the testing environment (perhaps, maybe, up to a point ... ) so testing might just boil down to "Oh looky here - there's a web site". But that might be a bit simplistic. Testing _will_ be required if the deployment updates a database - or is that part of migration? > > Also, question for django people: static files, as collected by > "manage.py collectstatic": in the repo, or not? > > > > > On 20 May 2013 13:03, Daniel Pope wrote: > It doesn't need to be SSH-based, there are orchestration > systems that are RPC or pubsub based (eg. Jenkins, > MCollective). > > As per the line between provisioning and deployment I've > argued in the past that provisioning should be for state that > spans many minor releases of an application. I also believe > it's preferable to require that deployment cannot use root > permissions, as this is less likely to impact independent > services hosted on the same machine. It also allows better > auditability - it prevents deployments making changes that are > rightly the domain of the provisioning system. > > On 20 May 2013 11:17, "MikedePlume" > wrote: > On Sat, 2013-05-18 at 17:41 +0100, Andy Robinson > wrote: > > Daniel, who are you disagreeing with? Everyone > here agrees on > > automation, I think. > > > > - Andy > > As a small user (one or two servers, one or two > packages to deploy): > > I started by doing a lot of typing of commands, got > bored with that, > moved on to command line scripts and I now use a bunch > of (python) > scripts, some fabric based, to manage the server farm > api and the ssh > stuff. I think the point is that the underlying > process is always going > to involve ssh and some server farm api and it is the > automation of all > of that where the cornucopia of choice (?) is > displayed. > > From a teaching perspective (that's what the book is > about, I guess) I'd > recognise the basics of ssh and so on and then build > up from there. The > basic principles behind testing a fabric deploy should > be extendable to > other environments, I'd have thought? > > I must say, I'd love to see something about testing > bash scripts, and, > perhaps, where to draw the line (if at all) between > server provisioning > and ci. > > Cheers > > Mike S. > > > > > > > > On 18 May 2013 12:24, Daniel Pope > wrote: > > > I thoroughly disagree with those who are saying > that for small > > > installations, it's less time-consuming to do > things manually. A > > > deployment/provisioning system gives you > reproducibility - an executable > > > record of how to re-create a server configuration > or re-run a deployment > > > without missing anything. The number of times I've > spent 20 minutes > > > wondering what I've missed all add up - that is > why it automation breaks > > > even so rapidly. > > > > > > Of course, you (or your employers) might have > other reasons to want > > > automation: > > > > > > - so that your deployment/provisioning is subject > to automated testing, > > > version control, code review, etc > > > - so that you can build replicas of production > servers for development, > > > testing or disaster recovery > > > - so that you can smash and rebuild a compromised > or faulty machine without > > > wasting time > > > - so that you can deploy a dozen times a day and > get features or fixes into > > > the hands of your users > > > > > > > > > On 17 May 2013 15:39, M.-A. Lemburg > wrote: > > >> > > >> On 16.05.2013 17:46, Andy Robinson wrote: > > >> > Speaking as a relatively obsolete dinosaur, I > would suggest that if > > >> > you are going to discuss specific deployment > practices, you start with > > >> > the most fundamental ones: SSH, the unix shell > and so on. > > >> > > > >> > We have had issues over the years with people > coming in and > > >> > introducing sexy new deployment tools, but > ultimately they all just > > >> > run unix commands. Anyone managing a web > application in the > > >> > non-microsoft world is ultimately depending on > this. Some key skills > > >> > (assuming a Linux/Mac/Unix-ish environment): > > >> > - know about SSH keys and logging into remote > machines > > >> > - know the basics of at least one command line > editor (e.g. vi) > > >> > - basic shell knowledge: environment > variables, testing for existence > > >> > of files and directories etc > > >> > - how to interact with your database from the > command line, if you use > > >> > one (including dump and restore) > > >> > - how your web server works: starting, > stopping, configuration files, > > >> > where log files live and how it talks to Python > > >> > > > >> > Fabric may be useful if you want to control > half a dozen machines from > > >> > your desktop, and it might add a lot of value > if you want to control a > > >> > hundred of them. But to update one server, you > deploy by logging into > > >> > it and then running commands or short scripts. > > >> > > > >> > For example, we have a 'demo site' we rebuild > pretty often that uses > > >> > Django, MYSQL, Celery and a few other things. > It runs on plain > > >> > vanilla Ubuntu machines we build ourselves. > The sequence is... > > >> > > > >> > 1. Log in via SSH > > >> > 2. CD to correct directory > > >> > 3. activate virtual environment > > >> > 4. stop any celery worker processes > > >> > 5. stop web server processes (* in our setup, > we leave Apache running) > > >> > 6. pull latest code from mercurial - both the > app, and 3-4 libraries > > >> > it depends on > > >> > 7. run a management command to rebuild the > database > > >> > 8. run a smallish in-place test suite > > >> > 9. restart celery workers > > >> > 10.restart web server > > >> > 11. log out > > >> > > > >> > All of this after the login and CD can be > handled by a shell script on > > >> > the path of the server, so you can just run a > command called something > > >> > like > > >> > ./update_server > > >> > > > >> > More realistically, we tend to end up with a > management shell script > > >> > called 'server' with a bunch of > commands/arguments like 'stop / start > > >> > / restart / update-code-in-staging / > copy-live-data-to-staging / > > >> > run-health-checks / swap-live-and-staging' and > so on. SSH can execute > > >> > remote commands like this just fine with the > right arguments, if > > >> > actually logging in is too tedious. > > >> > > > >> > Production sites are complex and all > different. You might want to do > > >> > instantaneous swaps from live to staging (and > be able to back out fast > > >> > if stuff goes wrong); to switch DNS so the > world is looking at another > > >> > server while you update one; you might have > large databases to copy or > > >> > migrate that need significant time; it may or > may not be acceptable to > > >> > lose sessions and have downtime; and so on. > > >> > > > >> > > > >> > It takes less time to learn the fundamentals > than you will spend > > >> > debugging why your fancy new deployment tool > stopped working after > > >> > some Python dependency upgrade somewhere. And > it is less likely that > > >> > your new hires will disagree if you stick with > the lowest common > > >> > denominator. > > >> > > >> Fully agreed. > > >> > > >> The new devops tools are nice when it comes to > managing farms > > >> of VMs, where each VM is setup in more or less > the same way, > > >> and you want to reduce repetitive tasks, but for > a typical > > >> setup with just a few VMs/servers it'll take you > longer to > > >> write and (most importantly) test your devops > scripts than > > >> it would to write a few scripts that you can run > via SSH on > > >> the servers. > > >> > > >> No matter how smart you make your devops scripts, > Murphy's law > > >> is inevitably going to strike and humans are so > much better at > > >> parsing weird intermittent error messages than > machines ... > > >> still, after all these years :-) > > >> > > >> -- > > >> Marc-Andre Lemburg > > >> eGenix.com > > >> > > >> Professional Python Services directly from the > Source (#1, May 17 2013) > > >> >>> Python Projects, Consulting and Support ... > http://www.egenix.com/ > > >> >>> mxODBC.Zope/Plone.Database.Adapter ... > http://zope.egenix.com/ > > >> >>> mxODBC, mxDateTime, mxTextTools ... > http://python.egenix.com/ > > >> > ________________________________________________________________________ > > >> 2013-05-07: Released mxODBC Zope DA 2.1.2 ... > http://egenix.com/go46 > > >> 2013-05-06: Released mxODBC 3.2.3 ... > http://egenix.com/go45 > > >> > > >> ::::: Try our mxODBC.Connect Python Database > Interface for free ! :::::: > > >> > > >> eGenix.com Software, Skills and Services GmbH > Pastor-Loeh-Str.48 > > >> D-40764 Langenfeld, Germany. CEO Dipl.-Math. > Marc-Andre Lemburg > > >> Registered at Amtsgericht Duesseldorf: > HRB 46611 > > >> > http://www.egenix.com/company/contact/ > > >> _______________________________________________ > > >> python-uk mailing list > > >> python-uk at python.org > > >> http://mail.python.org/mailman/listinfo/python-uk > > > > > > > > > > > > _______________________________________________ > > > python-uk mailing list > > > python-uk at python.org > > > http://mail.python.org/mailman/listinfo/python-uk > > > > > > > > > > > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > > > > > -- > ------------------------------ > Harry J.W. Percival > ------------------------------ > Twitter: @hjwp > Mobile: +44 (0) 78877 02511 > Skype: harry.percival > _______________________________________________ > python-uk mailing list > python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk From andy at reportlab.com Mon May 20 17:14:44 2013 From: andy at reportlab.com (Andy Robinson) Date: Mon, 20 May 2013 16:14:44 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: <1369060828.2123.67.camel@oberto> References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> <1369060828.2123.67.camel@oberto> Message-ID: On 20 May 2013 15:40, MikedePlume wrote: > Interesting that the subject veers towards system design. I use FCGI > myself and the restart can be entirely non-root. Provisioning then > includes adding the FCGI reference into the server config. > I'm glad we're not the only ones. We have been very happy with FCGI for 6 years now for exactly these reasons. -- Andy Robinson From andy at reportlab.com Mon May 20 17:12:09 2013 From: andy at reportlab.com (Andy Robinson) Date: Mon, 20 May 2013 16:12:09 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> Message-ID: On 20 May 2013 14:51, Harry Percival wrote: > Also, question for django people: static files, as collected by "manage.py > collectstatic": in the repo, or not? No. It's a deployment step. It's REALLY useful to have a small set of "functional tests" which can be run in a live web app. For example, for an online purchase process, you might have a selenium test which steps right through a purchase using a known "only for tests" account or credit card number. And it's an absolute lifesaver to have a bunch of CSS/JS URLs you hit to verify a 200, so you don't go live with half your javascript missing due to some static/media mixup in a settings file. Once you have something like this, you can quickly check that your deployment (whether to UAT, staging or live) actually worked. I don't know what this kind of testing is called though ;-) - Andy From daniele at vurt.org Mon May 20 17:30:46 2013 From: daniele at vurt.org (Daniele Procida) Date: Mon, 20 May 2013 16:30:46 +0100 Subject: [python-uk] Python/Django DevOps community and resources In-Reply-To: <3A170204-0DE4-4AF5-8ED7-A039F28D317A@voidspace.org.uk> References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> <18C4019E-5E9F-4219-A1A1-95A51CFFACF5@reynoldsfamily.org.uk> <3A170204-0DE4-4AF5-8ED7-A039F28D317A@voidspace.org.uk> Message-ID: <20130520153046.898664409@smtp.ntlworld.com> Hi folks. I've just got back from the most astounding DjangoCon Europe in Warsaw, where several of decided that the DevOps in the community needed more mutual support. So, we've set up #django-devops on irc.freenode.net, and for an email list. Do join us if this sort of thing is your concern, and please don't be put off by the "Django" bit. Anything that applies for Django will work more generally for anything in Python. If the name turns out to be too inappropriate or off-putting we can always change it later. Daniele From J.Gould at austinfraser.com Mon May 20 18:31:12 2013 From: J.Gould at austinfraser.com (Jon Gould) Date: Mon, 20 May 2013 16:31:12 +0000 Subject: [python-uk] Python/Django DevOps community and resources In-Reply-To: <20130520153046.898664409@smtp.ntlworld.com> References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> <18C4019E-5E9F-4219-A1A1-95A51CFFACF5@reynoldsfamily.org.uk> <3A170204-0DE4-4AF5-8ED7-A039F28D317A@voidspace.org.uk> <20130520153046.898664409@smtp.ntlworld.com> Message-ID: While we're talking groups and meetups, I'm looking at dates to arrange the next DJUGL. DJUGL is the Django User Group London which is a social meetup consisting of Beer, Pizza and Tech Talks Are there any conflicting London tech nights on the 11th or 18th June? Regards Jon Gould | Team Leader Austin Fraser Limited | Digital Contracts Tel: 0118 952 0153 | Skype: JonWebJobs | Twitter: @JonWebJobs ? -----Original Message----- From: python-uk [mailto:python-uk-bounces+j.gould=austinfraser.com at python.org] On Behalf Of Daniele Procida Sent: 20 May 2013 16:31 To: UK Python Users Subject: [python-uk] Python/Django DevOps community and resources Hi folks. I've just got back from the most astounding DjangoCon Europe in Warsaw, where several of decided that the DevOps in the community needed more mutual support. So, we've set up #django-devops on irc.freenode.net, and for an email list. Do join us if this sort of thing is your concern, and please don't be put off by the "Django" bit. Anything that applies for Django will work more generally for anything in Python. If the name turns out to be too inappropriate or off-putting we can always change it later. Daniele _______________________________________________ python-uk mailing list python-uk at python.org http://mail.python.org/mailman/listinfo/python-uk ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________ From peter.inglesby at gmail.com Mon May 20 20:52:26 2013 From: peter.inglesby at gmail.com (Peter Inglesby) Date: Mon, 20 May 2013 19:52:26 +0100 Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> Message-ID: > > It's REALLY useful to have a small set of "functional tests" which can > be run in a live web app. For example, for an online purchase > process, you might have a selenium test which steps right through a > purchase using a known "only for tests" account or credit card number. > And it's an absolute lifesaver to have a bunch of CSS/JS URLs you hit > to verify a 200, so you don't go live with half your javascript > missing due to some static/media mixup in a settings file. > > Once you have something like this, you can quickly check that your > deployment (whether to UAT, staging or live) actually worked. > > I don't know what this kind of testing is called though ;-) > Sounds like "smoke testing" to me. From jjl at pobox.com Mon May 20 22:07:43 2013 From: jjl at pobox.com (John Lee) Date: Mon, 20 May 2013 21:07:43 +0100 (BST) Subject: [python-uk] CI and deployment (was Re: Suggestions / best practices for deployment) In-Reply-To: <4AED6FAB-15BE-456B-97FA-A43002EEE2ED@cavallinux.eu> References: <51936F5A.1040901@tangentlabs.co.uk> <4AED6FAB-15BE-456B-97FA-A43002EEE2ED@cavallinux.eu> Message-ID: On Sat, 18 May 2013, Antonio Cavallo wrote: > I don't know any scenario like that: usually dependencies are frozen > (and tested) separately in a "release". It makes the whole process > simpler. I guess you missed this part? [john wrote:] > (even though production is pinned to old releases, you probably > want to find out about the breakage early because it might get picked up > later on in production) > Possibly they wanted to test how far they can push TDD, to be able to do > testing on the entire code (apps + libraries) using the development > branch (all commits done on it), api changes in the clients breaking the > servers? This is for integration testing, yes. I don't think TDD is relevant, because their system is useful regardless of whether you do TDD. The claimed innovation is not that they invented automated integration testing (!), but that they only rebuild modules where needed, so that faults are picked up sooner. I haven't seen open source code that makes it easy to do that, and I would likely use an open source project that did, because I have specific uses for it (which I briefly described in my post). > I hope you won't find *any* CI doing that: that'd be very "code > specific" nothing that a CI should be aware of :D Sorry, I probably don't understand this. Do you mean that CI should not be used for integration testing (and perhaps also that it shouldn't be used for testing deployment processes)? Why is that? I've found it useful for both of those things. Deployment is something that can go wrong, so it is useful to test it -- especially when it gets complicated. I have even seen CI used for testing upgrade of specific databases and associated configuration code, and that was also useful. John From jjl at pobox.com Mon May 20 22:16:59 2013 From: jjl at pobox.com (John Lee) Date: Mon, 20 May 2013 21:16:59 +0100 (BST) Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: <1369045053.2123.28.camel@oberto> References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> Message-ID: On Mon, 20 May 2013, MikedePlume wrote: > I must say, I'd love to see something about testing bash scripts, and, Something like this: child = subprocess.Popen(["find", deploy_root, "-name", "*.sh"]) out, err = child.communicate() bad = out.splitlines() self.assertEqual(len(bad), 0) ;-) John From jjl at pobox.com Mon May 20 22:23:49 2013 From: jjl at pobox.com (John Lee) Date: Mon, 20 May 2013 21:23:49 +0100 (BST) Subject: [python-uk] Suggestions / best practices for deployment In-Reply-To: References: <048D92DC-F16B-4836-88B9-94E85D2FF4E6@gmail.com> <51964118.4040402@egenix.com> <1369045053.2123.28.camel@oberto> Message-ID: On Mon, 20 May 2013, Andy Robinson wrote: > be run in a live web app. For example, for an online purchase > process, you might have a selenium test which steps right through a > purchase using a known "only for tests" account or credit card number. [...] > I don't know what this kind of testing is called though ;-) Expensive?-) John From tibs at tibsnjoan.co.uk Tue May 28 20:49:08 2013 From: tibs at tibsnjoan.co.uk (Tony Ibbs) Date: Tue, 28 May 2013 19:49:08 +0100 Subject: [python-uk] Next CamPUG meeting: Tue 4th June 2013 Message-ID: <04DDF265-3720-4831-8FCD-BC121F37E64A@tibsnjoan.co.uk> From the Cambridge and East Anglian Python Group google group: The next meeting will be Tuesday 4th Junel, 7.30pm at RealVNC (http://tinyurl.com/realvncoffices). We normally stop about 9.30pm, and go on to the pub. This will be a "doing stuff" meeting. Specific ideas of "stuff" are always welcome. Meetings after that should be: ? Tuesday 2nd July, a talks meeting ? Tuesday 6th August, another doing stuff meeting ? Tuesday 3rd September, another talks meeting Tibs From whykay at python.ie Wed May 29 12:33:09 2013 From: whykay at python.ie (Vicky Lee - Python Ireland) Date: Wed, 29 May 2013 11:33:09 +0100 Subject: [python-uk] PyCon Ireland Early Bird tickets ending in just over 2 weeks Message-ID: Hi All, When: 12th - 15th October 2013 Where: Burlington Hotel (conference) / College of Computer Training (sprints), Dublin Early Bird tickets are still available. Just over two weeks left! Ends *June 16th*. Buy yours at http://python.ie/pycon/ If you are interested in any of the sprints, you can register via http://python.ie/pycon/ as well. Call for Proposals and Sponsorsare still open. If you have any questions, please contact pycon at python.ie. Cheers, /// Vicky (PyCon Ireland co-Chair) Python Ireland co-Chair / Treasurer EuroPython Board PSF member -------------- next part -------------- An HTML attachment was scrubbed... URL: From rachid.belaid at gmail.com Wed May 29 15:52:41 2013 From: rachid.belaid at gmail.com (Rachid Belaid) Date: Wed, 29 May 2013 14:52:41 +0100 Subject: [python-uk] Second London Pyramid Meetup Message-ID: Hello, Thank you to everyone who came to the first Pyramid meetup. This month we'd like continue sharing and learning about Pyramid with some fresh talks, and we are also prepared to welcome any newcomers! The next meetup will be on *Tuesday, June 4, 2013*: http://www.meetup.com/The-London-Pyramid-Group/events/119944802/ We are still looking for at least one more speaker so let me know if you would be interested to do a talk about a Pyramid related subject: Pyramid, Pylons, SQLAlchemy, Beaker/DogPile (cache/session) ... There's no need to already be a pyramid user to join us. If you're interested in this framework, need to get started or simply curious about a framework you've never heard about then of course you are welcome to join us. Rach *Previous Meetup slides: * *View Lookup*: http://www.slideshare.net/rachbelaid/pyramid-views-20820325 *Routing and traversal*: https://dl.dropboxusercontent.com/u/5469585/PyramidLondon-Routing.pdf -- Rach Belaid @rachbelaid -------------- next part -------------- An HTML attachment was scrubbed... URL: From samphippen at googlemail.com Thu May 30 15:03:00 2013 From: samphippen at googlemail.com (Sam Phippen) Date: Thu, 30 May 2013 14:03:00 +0100 Subject: [python-uk] upcoming python dojo Message-ID: <4288410E-0997-44D7-BCCE-724C2BC90912@googlemail.com> Hi Guys, I saw the tweets for the upcoming python dojo, at the last one I went to there was a lightning talk about bit torrent. There was some interest in bit coin, I'd be happy to do a lightning talk about how bit coin works technically if people are interested. Thanks -- Sam Phippen From Ben.Curwood at hogarthww.com Thu May 30 15:26:12 2013 From: Ben.Curwood at hogarthww.com (Ben Curwood) Date: Thu, 30 May 2013 13:26:12 +0000 Subject: [python-uk] Python/Django at Hogarth Worldwide Message-ID: Afternoon Everyone, Here at Hogarth, we are looking for Python/Django Developers to join our ever-evolving technology department. We run large application services that interact with global agencies and advertisers to produce, store and distribute advertisements worldwide. Here at Hogarth we pride ourselves in keeping on the pulse of the latest developments to make the work we do and how we do it the most exciting, Agile and dynamic environment to work in. We are friendly, we are fast moving, we have a casual dress code and we like a drink. What you can hopefully bring to us: * Fluency in Python 2.5+ and experience building/contributing to large Django web applications * Good understanding of RESTful web services and developing/extending a REST API ideally * Experience of real-time push/long-polling tech, such as node.js and websockets * Experience working with SVN and GIT * Experience working in an Agile/SCRUM environment * Strong TDD experience What we can offer you: * A kick ass Mac * Nice office in the middle of London, Clerkenwell (or Covent Garden - we're moving soon) * Cool colleagues, who you will learn from * Some excellent projects for you to do your best work ever * Relaxed, fun environment, conducive to doing good work * The opportunity to learn about new methods and technologies Open Positions: Python/Django Developer Senior Python/Django Developer Lead Python/Django Developer Developer in Test (TDD/Automation specialist) Contact us if you are interested in talking further! -- Ben Curwood Hogarth Worldwide Limited Direct: +44 207 2406400 Mobile: +44 7768 557 654 Linkedin: http://uk.linkedin.com/in/bencurwood Twitter: http://www.twitter.com/ben_curwood Are you a freelancer? Join our Linkedin group: Hogarth London Freelance Group www.hogarthww.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ntoll at ntoll.org Thu May 30 16:17:59 2013 From: ntoll at ntoll.org (Nicholas H.Tollervey) Date: Thu, 30 May 2013 15:17:59 +0100 Subject: [python-uk] upcoming python dojo In-Reply-To: <4288410E-0997-44D7-BCCE-724C2BC90912@googlemail.com> References: <4288410E-0997-44D7-BCCE-724C2BC90912@googlemail.com> Message-ID: <51A75F97.6070703@ntoll.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 30/05/13 14:03, Sam Phippen wrote: > Hi Guys, > > I saw the tweets for the upcoming python dojo, at the last one I > went to there was a lightning talk about bit torrent. There was > some interest in bit coin, I'd be happy to do a lightning talk > about how bit coin works technically if people are interested. > > Thanks -- Sam Phippen > _______________________________________________ python-uk mailing > list python-uk at python.org > http://mail.python.org/mailman/listinfo/python-uk > I think that'd be a great idea. I've cc'd Gautier (who is cat herding the event). I guess we're looking at between 5-10 minutes worth of material at the most. N. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJRp1+XAAoJEP0qBPaYQbb6LWcIAKN2NmVMxwXSf+rElQ6gXv9W qVW0OP3MZbtb+Ibk7wdwlS/uCFRz025ATBKs/PAb/j5cgxlbZRiY3tazE9n7p++D Ye6PVxf2TaIf2HLlfsYmRn5h62k8MtmF3WLylqnNQxkdlRUW7hSYEB+AfKP+iuqP YOPu4dGd6HG1162RgoKCkShgE5Lj2unw3xz7uyNYauWBsetnqVkQ+T8Kyoo8jzb1 jBMC91xzB9Yh1OHmr94seAG/5IvJ/yhbdKKpoxUD5vsOhFW8isL+nTUv5DnYhSrk dm7IZmE9pAWq+QNiDzlbZPDht00vxYG4gOTkNeEswVc/99BpyLpizYx0qJCAvYQ= =8VrT -----END PGP SIGNATURE----- From ghayoun at gmail.com Thu May 30 16:24:55 2013 From: ghayoun at gmail.com (Gautier HAYOUN) Date: Thu, 30 May 2013 15:24:55 +0100 Subject: [python-uk] London Python Code Dojo is happening next Thursday Message-ID: Good afternoon Pythonistas, next week on Thursday will be the first Thursday (06/06) of June and that means London Python Code Dojo! We are moving to Bank of America in Canary Wharf this time so come try our new host hospitality by grabbing some tickets before they are all gone : https://ldnpydojo.eventwax.com/london-python-code-dojo-season-4-episode-10 Given the nature of our host, Nicholas is suggesting "to think up some bank/finance/economic-related problems for the dojo to solve. Implement a Monte Carlo / Las Vegas algorithm? (Look 'em up on Wikipedia) Something Bitcoin related..? A simple "god" game that requires you manage an economy?" If you can think of problems in the same vein please share them with us when you book your ticket. If you need any more information, contact the team via Twitter: @ldnpydojo or via email team at ldnpydojo.org.uk . Look forward to seeing you all next Thursday. Best, Gautier From daniele at vurt.org Thu May 30 22:43:11 2013 From: daniele at vurt.org (Daniele Procida) Date: Thu, 30 May 2013 21:43:11 +0100 Subject: [python-uk] Workshop: Don't Be Afraid to Commit, Cardiff Message-ID: <20130530204311.419454090@smtp.ntlworld.com> I'm running a Don't Be Afraid to Commit in Cardiff weekend after next, under the auspices of the excellent Cardiff Dev Workshop, . It's a workshop/tutorial for Python/Django developers who would like to contribute to the projects they use, but need more grounding in some of the tools required. I ran this at the DjangoCon Europe sprints in Warsaw a couple of weeks ago, and it seemed to work pretty well there, See for more information. The workshop will take participants through the complete cycle of identifying a simple issue in a Django or Python project, writing a patch with tests and documentation, and submitting it. The workshop is aimed at the first-time committer. Very little experience is required: It's free to anyone who'd like to attend, but you need to register because places are limited: . Regards, Daniele