From bnansel at bigpond.net.au Thu May 10 05:20:48 2007 From: bnansel at bigpond.net.au (Robert Nansel) Date: Thu, 10 May 2007 12:50:48 +0930 Subject: [sapug] New to Adelaide and Python ... Message-ID: <8916A91D-87BD-4BE7-ABBF-FCCE0464EA37@bigpond.net.au> ... though I am a grizzled embedded systems developer, so I'm not a novice programmer (just a bit antiquated). I migrated from the USA with my family to Adelaide September last, and I'm looking for other folks who are interested in embedded systems and mechatronics. I'm a Robotics Engineer (or so it says on my Bachelor's diploma), and I'm exploring what I can do with Python in that area. I'm learning Python 2.5, under OSX for the time being, and I'd appreciate any tips on good tutorials, cookbooks, example code to look at, online or in books. I have ORA's Learning Python by Lutz & Ascher, but it's the old first edition, so it only covers up to Python 1.5. Anyway, I eventually I hope to apply Python to the more resource- constrained environments typical of embedded systems (single-chip microcontrollers, limited RAM, no hard disk, etc.). Does anyone on this list do such work? Regards, -Bobby Nansel From bnansel at bigpond.net.au Thu May 10 05:20:48 2007 From: bnansel at bigpond.net.au (Robert Nansel) Date: Thu, 10 May 2007 12:50:48 +0930 Subject: [sapug] New to Adelaide and Python ... Message-ID: <8916A91D-87BD-4BE7-ABBF-FCCE0464EA37@bigpond.net.au> ... though I am a grizzled embedded systems developer, so I'm not a novice programmer (just a bit antiquated). I migrated from the USA with my family to Adelaide September last, and I'm looking for other folks who are interested in embedded systems and mechatronics. I'm a Robotics Engineer (or so it says on my Bachelor's diploma), and I'm exploring what I can do with Python in that area. I'm learning Python 2.5, under OSX for the time being, and I'd appreciate any tips on good tutorials, cookbooks, example code to look at, online or in books. I have ORA's Learning Python by Lutz & Ascher, but it's the old first edition, so it only covers up to Python 1.5. Anyway, I eventually I hope to apply Python to the more resource- constrained environments typical of embedded systems (single-chip microcontrollers, limited RAM, no hard disk, etc.). Does anyone on this list do such work? Regards, -Bobby Nansel From bnansel at bigpond.net.au Thu May 10 05:20:48 2007 From: bnansel at bigpond.net.au (Robert Nansel) Date: Thu, 10 May 2007 12:50:48 +0930 Subject: [sapug] New to Adelaide and Python ... Message-ID: <8916A91D-87BD-4BE7-ABBF-FCCE0464EA37@bigpond.net.au> ... though I am a grizzled embedded systems developer, so I'm not a novice programmer (just a bit antiquated). I migrated from the USA with my family to Adelaide September last, and I'm looking for other folks who are interested in embedded systems and mechatronics. I'm a Robotics Engineer (or so it says on my Bachelor's diploma), and I'm exploring what I can do with Python in that area. I'm learning Python 2.5, under OSX for the time being, and I'd appreciate any tips on good tutorials, cookbooks, example code to look at, online or in books. I have ORA's Learning Python by Lutz & Ascher, but it's the old first edition, so it only covers up to Python 1.5. Anyway, I eventually I hope to apply Python to the more resource- constrained environments typical of embedded systems (single-chip microcontrollers, limited RAM, no hard disk, etc.). Does anyone on this list do such work? Regards, -Bobby Nansel From bnansel at bigpond.net.au Thu May 10 05:20:48 2007 From: bnansel at bigpond.net.au (Robert Nansel) Date: Thu, 10 May 2007 12:50:48 +0930 Subject: [sapug] New to Adelaide and Python ... Message-ID: <8916A91D-87BD-4BE7-ABBF-FCCE0464EA37@bigpond.net.au> ... though I am a grizzled embedded systems developer, so I'm not a novice programmer (just a bit antiquated). I migrated from the USA with my family to Adelaide September last, and I'm looking for other folks who are interested in embedded systems and mechatronics. I'm a Robotics Engineer (or so it says on my Bachelor's diploma), and I'm exploring what I can do with Python in that area. I'm learning Python 2.5, under OSX for the time being, and I'd appreciate any tips on good tutorials, cookbooks, example code to look at, online or in books. I have ORA's Learning Python by Lutz & Ascher, but it's the old first edition, so it only covers up to Python 1.5. Anyway, I eventually I hope to apply Python to the more resource- constrained environments typical of embedded systems (single-chip microcontrollers, limited RAM, no hard disk, etc.). Does anyone on this list do such work? Regards, -Bobby Nansel From toojays at toojays.net Thu May 10 14:12:28 2007 From: toojays at toojays.net (John Steele Scott) Date: Thu, 10 May 2007 12:12:28 +0000 (UTC) Subject: [sapug] New to Adelaide and Python ... References: <8916A91D-87BD-4BE7-ABBF-FCCE0464EA37@bigpond.net.au> Message-ID: On Thu, 10 May 2007 12:50:48 +0930, Robert Nansel wrote: > Anyway, I eventually I hope to apply Python to the more resource- > constrained environments typical of embedded systems (single-chip > microcontrollers, limited RAM, no hard disk, etc.). Does anyone on > this list do such work? Welcome Bobby, I'm doing a fair bit of microcontroller work (no robotics, just dataloggers mostly), but the biggest device I'm using only has 24kB of flash and 2kB of RAM. Although I use Python regularly during development, I've never contemplated putting a Python VM on the micro. Is that what you meant? I mostly use Python for quick & dirty datalogger<->PC communications, testing, and the occasional bit of code generation (i.e. writing Python programs which spit out C code for me). regards, John From michael.cohen at netspeed.com.au Thu May 10 14:33:39 2007 From: michael.cohen at netspeed.com.au (Michael Cohen) Date: Thu, 10 May 2007 22:33:39 +1000 Subject: [sapug] New to Adelaide and Python ... In-Reply-To: References: <8916A91D-87BD-4BE7-ABBF-FCCE0464EA37@bigpond.net.au> Message-ID: <20070510123339.GA12613@OpenWrt> Bobby, I once compiled python for openwrt based wgt634u routers. I think it was just under a mb but its a heavy price to pay when your flash is 4mb in total. Might be ok for some things though. I can dig up the build scripts if you like. The thing about python is that the interpreter is quite small, but by the time you add up all the standard library modules it can become very large - thats when touch and find are your friends (basically set atime to 0, run your unit tests and then use find to work out which modules were touched and include them in). Python has a great feature where modules can be used directly from a zip file which keeps the size down. But ultimately with use of squashfs makes little difference i suppose. Michael. On Thu, May 10, 2007 at 12:12:28PM +0000, John Steele Scott wrote: > On Thu, 10 May 2007 12:50:48 +0930, Robert Nansel wrote: > > Anyway, I eventually I hope to apply Python to the more resource- > > constrained environments typical of embedded systems (single-chip > > microcontrollers, limited RAM, no hard disk, etc.). Does anyone on > > this list do such work? > > Welcome Bobby, > > I'm doing a fair bit of microcontroller work (no robotics, just dataloggers > mostly), but the biggest device I'm using only has 24kB of flash and 2kB of > RAM. Although I use Python regularly during development, I've never > contemplated putting a Python VM on the micro. Is that what you meant? > > I mostly use Python for quick & dirty datalogger<->PC communications, > testing, and the occasional bit of code generation (i.e. writing Python > programs which spit out C code for me). > > regards, > > John > > _______________________________________________ > sapug mailing list > sapug at python.org > http://mail.python.org/mailman/listinfo/sapug > From twegener at fastmail.fm Thu May 10 15:32:54 2007 From: twegener at fastmail.fm (Tim Wegener) Date: Thu, 10 May 2007 23:02:54 +0930 Subject: [sapug] New to Adelaide and Python ... In-Reply-To: <8916A91D-87BD-4BE7-ABBF-FCCE0464EA37@bigpond.net.au> References: <8916A91D-87BD-4BE7-ABBF-FCCE0464EA37@bigpond.net.au> Message-ID: <1178803975.1363.28.camel@bergamot.wegener.org> On Thu, 2007-05-10 at 12:50 +0930, Robert Nansel wrote: > I'm learning Python 2.5, under OSX for the time being, and I'd > appreciate any tips on good tutorials, cookbooks, example code to > look at, online or in books. I have ORA's Learning Python by Lutz & > Ascher, but it's the old first edition, so it only covers up to > Python 1.5. I've been working with Python regularly for about the last five years. Over that time I haven't found the need to buy any books on the subject, as there are ample resources online and they tend to be more concise and up to date. Here is my recommended reading list: The "Official" Python Tutorial by Guido van Rossum (the author of Python) (This should get you up to speed in a day.) http://docs.python.org/tut/tut.html The standard library module index: http://docs.python.org/modindex.html The Python Style Guide also by GvR (style matters!) http://www.python.org/doc/essays/styleguide.html The Python Quick Reference (this is the main place I turn to these days; everything on one page and shows differences between Python versions) http://rgruet.free.fr/PQR25/PQR2.5.html Dive into Python (if you want some more tutorials) http://diveintopython.org/ And don't forget: $ python >>> import this Beyond this, comp.lang.python is a good place to search and ask questions. (Especially keep an eye out for posts by any of the 'bots': Tim Peters, Alex Martelli, Fredrik Lundh.) I have had a flick through Mark Lutz's 'Programming Python' and also 'Learning Python' that you mentioned. These felt a bit bloated to me. > Anyway, I eventually I hope to apply Python to the more resource- > constrained environments typical of embedded systems (single-chip > microcontrollers, limited RAM, no hard disk, etc.). Does anyone on > this list do such work? I know of RFID readers that are embedded devices that run Python on Linux. Not sure what the resources are in terms of RAM/CPU, perhaps in the order of 8MB? I came across this on the web one day, Python on a chip: http://www.telit.co.it/product.asp?productId=32 Also, Python can run on Nokia Series 60 phones, ARM based PDAs, Intel PDAs, Palm, etc. I imagine you are talking about stricter resource constraints than these have, though. You may also find this interesting: http://students.ceid.upatras.gr/~sxanth/pyvm/ What embedded applications do you have in mind? Tim From twegener at fastmail.fm Fri May 11 03:23:07 2007 From: twegener at fastmail.fm (Tim Wegener) Date: Fri, 11 May 2007 10:53:07 +0930 Subject: [sapug] New to Adelaide and Python ... In-Reply-To: <1178803975.1363.28.camel@bergamot.wegener.org> References: <8916A91D-87BD-4BE7-ABBF-FCCE0464EA37@bigpond.net.au> <1178803975.1363.28.camel@bergamot.wegener.org> Message-ID: <20070511105307.5fc7661a.twegener@fastmail.fm> Here is another interesting Python implementation for embedded systems: http://pymite.python-hosting.com/ PyMite is a flyweight Python interpreter written from scratch to execute on 8-bit and larger microcontrollers with resources as limited as 64 KiB of program memory (flash) and 4 KiB of RAM. PyMite supports a subset of the Python 2.5 syntax and can execute a subset of the Python 2.5 bytecodes. PyMite can also be compiled, tested and executed on a desktop computer. Tim From sales at ebmpl.com.au Sat May 26 06:39:09 2007 From: sales at ebmpl.com.au (Eric March) Date: Sat, 26 May 2007 14:09:09 +0930 Subject: [sapug] Programme to update mysql table Linux redhat Message-ID: <20070526050559.3BD661E4007@bag.python.org> Anyone interested in writing a small programme to read a fixed width text file and update an sql table? If so ring me to discuss cost. Thanks Eric March Electronic Business Management Pty Ltd PO Box 475 Hahndorf SA 5245 Ph 08 8388 1991 Fax 08 8388 1998 Mbl 0418 418 995 From dt-sapug at handcraftedcomputers.com.au Tue May 29 03:33:53 2007 From: dt-sapug at handcraftedcomputers.com.au (Daryl Tester) Date: Tue, 29 May 2007 11:03:53 +0930 Subject: [sapug] Python Resource: Testing In Python Mailing List. Message-ID: <465B8301.3020505@handcraftedcomputers.com.au> Y'all. I stumbled across this the other month - the "Testing In Python" mailing list. For all you Mock Objectivists and people who, like horror, test the code you write. There's even some interesting snippets from Jim Fulton (*The* Zope Architect) on their testing strategies. Ye obligatory gmane archive: -- Regards, Daryl Tester "My god, it's full of 'car's!" - "LISP", http://xkcd.com/c224.html From chris at inetd.com.au Wed May 30 03:36:01 2007 From: chris at inetd.com.au (Chris Foote) Date: Wed, 30 May 2007 11:06:01 +0930 (CST) Subject: [sapug] Python Resource: Testing In Python Mailing List. In-Reply-To: <465B8301.3020505@handcraftedcomputers.com.au> References: <465B8301.3020505@handcraftedcomputers.com.au> Message-ID: On Tue, 29 May 2007, Daryl Tester wrote: > I stumbled across this the other month - the "Testing In Python" > mailing list. For all you Mock Objectivists and people who, like > horror, test the code you write. There's even some interesting > snippets from Jim Fulton (*The* Zope Architect) on their testing > strategies. > > I've toyed with PyUnit a little bit (http://pyunit.sourceforge.net) but I don't really understand test driven development. To me it seems like you'd spend far too much extra time getting your code to fit the tests than getting real work done, especially with projects than involve lots of external components like databases, distributed procedures and actions that take place in the background using threads. Maybe I'm missing a big piece of the testing puzzle :-( Having code not break when getting modified by herds of software engineers would have to be a good reason to use it, but aren't its benefits very limited for one or two man projects ? -- Chris Foote Inetd Pty Ltd T/A HostExpress Web: http://www.hostexpress.com.au Blog: http://www.hostexpress.com.au/drupal/chris Phone: (08) 8410 4566 From twegener at fastmail.fm Wed May 30 15:22:34 2007 From: twegener at fastmail.fm (Tim Wegener) Date: Wed, 30 May 2007 22:52:34 +0930 Subject: [sapug] Python Resource: Testing In Python Mailing List. In-Reply-To: References: <465B8301.3020505@handcraftedcomputers.com.au> Message-ID: <1180531355.3504.39.camel@bergamot.wegener.org> On Wed, 2007-05-30 at 11:06 +0930, Chris Foote wrote: > On Tue, 29 May 2007, Daryl Tester wrote: > > > I stumbled across this the other month - the "Testing In Python" > > mailing list. For all you Mock Objectivists and people who, like > > horror, test the code you write. There's even some interesting > > snippets from Jim Fulton (*The* Zope Architect) on their testing > > strategies. > > > > > > I've toyed with PyUnit a little bit (http://pyunit.sourceforge.net) > but I don't really understand test driven development. Here are some other testing frameworks worth looking at: Doctest is in the standard library. Writing tests is simply a matter of writing dialogs of what would happen at the interactive python prompt and put that in the function docstring. http://docs.python.org/lib/module-doctest.html Py.Test is an alternative to PyUnit. Writing tests is as simple as writing assert statements. http://wiki.python.org/moin/PyTest Nose is another test system along the lines of Py.Test: http://somethingaboutorange.com/mrl/projects/nose/ > To me it seems like you'd spend far too much extra time getting your code > to fit the tests than getting real work done, especially with projects > than involve lots of external components like databases, distributed > procedures and actions that take place in the background using threads. > Maybe I'm missing a big piece of the testing puzzle :-( For external components it is often useful to use stubs/mocks etc instead. Stubs/mocks are basically fake versions of these components where you can easily control their behaviour. This can help make test suites run faster, and makes it easier to control behaviour that would be non-deterministic in the final application. > Having code not break when getting modified by herds of software engineers > would have to be a good reason to use it, but aren't its benefits very > limited for one or two man projects ? All code needs to get debugged. When you debug code you need to prove to yourself that it is working where it wasn't before. You might as well formalise proof as a test. Doing this up front clarifies the problem, and shows whether the bug, as defined by the test, has been fixed. A comprehensive set of tests is essentially an executable spec. Any untested behaviour is undefined in terms of the spec. Even for projects implemented by one or two people the tests are very valuable as spec, regression testing and as a form of API documentation that is guaranteed to be in sync with the implementation. In my experience a good rule of thumb is: if it hasn't been tested it's almost definitely broken. Mind you, it's easy to say all of this, but it takes a bit of discipline and faith in the process to launch into the test-first approach. Tim From michael at msdavies.net Wed May 30 15:35:38 2007 From: michael at msdavies.net (Michael Davies) Date: Wed, 30 May 2007 23:05:38 +0930 Subject: [sapug] Python Resource: Testing In Python Mailing List. In-Reply-To: References: <465B8301.3020505@handcraftedcomputers.com.au> Message-ID: On 30/05/07, Chris Foote wrote: > I've toyed with PyUnit a little bit (http://pyunit.sourceforge.net) > but I don't really understand test driven development. > > To me it seems like you'd spend far too much extra time getting your code > to fit the tests than getting real work done, especially with projects > than involve lots of external components like databases, distributed > procedures and actions that take place in the background using threads. > Maybe I'm missing a big piece of the testing puzzle :-( > > Having code not break when getting modified by herds of software engineers > would have to be a good reason to use it, but aren't its benefits very > limited for one or two man projects ? Well, I'd say no :-) Your tests also define your requirements in a far more concrete way than any externally developed and updated spec. While the code might be the definitive definition of what your software does, the tests document implicit behavior, moving that behaviour to become an explicit requirement, and hence persistent through subsequent modification (even by yourself - who can remember the code they wrote last month? Or even last _week_!?!?) I'd also say that TDD also encourages you to only write the code you need to. Far too often we're asked to build a rope bridge over a ravine, and instead we build a harbor bridge :) TDD makes us think about what we're building. TDD encourages us to write code that is testable - meaning that when a critical bug does come in, you don't have to resort to a debugger - it's going to be more likely to be able to write a test to catch that heisenbug (corollary: Having unit tests for a multi-threaded system is far better than trying to use a debugger in a multi-threaded system) TDD also improves your productivity - since you're not trying to come up with the grand unified theory of everything, but rather writing only enough code to satisfy the next test, you get more done. The demotivation of hard problems is dissolved into small tests with small solutions. Regarding the problem of interfacing to external databases and network resources, making use of mock objects can alleviate this. It's still more work, but helps us to solidify interfaces and understand the problem better. Eat your greens, they're good for you :) Of course, TDD is much easier when starting a project from scratch. I have a 30KSLOC non-python code base right now where we're trying to retrofit unit tests and implement TDD practices going forward. Let's say it's not fun[1]. But we're already experienced some benefits - refactoring has been easier than expected for adding new functionality. So to summarise, I'm a convert. It's TDD for me! Michael... [1] But the testing framework is partially in Python, so there's _some_ fun :-) -- Michael Davies "Do what you think is interesting, do somthing that michael at the-davies.net you think is fun and worthwhile, because otherwise http://michaeldavies.org you won't do it well anyway." -- Brian Kernighan From dt-sapug at handcraftedcomputers.com.au Thu May 31 06:24:12 2007 From: dt-sapug at handcraftedcomputers.com.au (Daryl Tester) Date: Thu, 31 May 2007 13:54:12 +0930 Subject: [sapug] Python Resource: Testing In Python Mailing List. In-Reply-To: References: <465B8301.3020505@handcraftedcomputers.com.au> Message-ID: <465E4DEC.10403@handcraftedcomputers.com.au> Chris Foote wrote: > I've toyed with PyUnit a little bit (http://pyunit.sourceforge.net) > but I don't really understand test driven development. I'm still split on its merits, given the amount of bug free code that I write :-), but it's still something I'm coming to grips with (despite my surname, it ain't second nature). About a year ago I did write some fairly baroque "accounting code" that benefited a lot from writing the test code up front, but then (to me) that's more nailing its specification than anything else (although I did add corner test cases later as I found them). But that was an independant module that had little in the way of external dependencies, and I think that's the easiest of cases. Not all problems decompose so easily. > To me it seems like you'd spend far too much extra time getting your code > to fit the tests than getting real work done, especially with projects > than involve lots of external components like databases, distributed > procedures and actions that take place in the background using threads. It's at this point where I come unstuck too. You are right in that test firsting your code does modify its structure, and presumably once you've had enough practice then it becomes easier, but I'm still grappling with getting mock objects inserted into strategically placed positions in "substantial" code (and here I'd be looking at more end to end than unit testing). My problem (experience?) here has been that I try to test only behaviour (more along the interface lines) than the implementation, so I like to keep my tests only loosely coupled to the code. Once the tests start getting into the "OK, I need to mock all the objects which cause side effects, so I'll need a postgres and ldap object, and patch in this replacement module" then you start spending too much time keeping the tests "in tune" with the implementation. Doctest'ing appears to be more interface approached, although I haven't had as much experience with it as unit testing. Retroactively fitting tests seems to be a level of hardness again, so I think the decision to do substantial unit testing has to be made up front (certainly hard enough that someone's devoted a book to the topic :-). > Having code not break when getting modified by herds of software engineers > would have to be a good reason to use it, but aren't its benefits very > limited for one or two man projects ? I think it depends on the nature of the project. The "spec. nailing" example I did was where I first got the tickle that it would work (and in its subsequent refactorings which were done a few months later, when implementation details weren't so fresh in my mind). For a while I've wanted to rewrite one of my more substantial Python modules to make it more testable, but I think my approach is wrong - I'm too bogged down by its current implementation (thinking "How will I test this particular component") to figure out where to start (and throwing away perfectly working code doesn't sound like much of a win either). -- Regards, Daryl Tester "My god, it's full of 'car's!" - "LISP", http://xkcd.com/c224.html