From GConiglio at ene.com Fri Jun 1 15:23:14 2012 From: GConiglio at ene.com (Coniglio, Gregory) Date: Fri, 1 Jun 2012 09:23:14 -0400 Subject: [CentralOH] Way off topic, but any Access programmers available? In-Reply-To: <803A920856B5A947AB0F0FCC2A935A9E1575D08F@USDEN3EXMB003.na.aecomnet.com> References: <008601cd3da5$f170eba0$d452c2e0$@com> <803A920856B5A947AB0F0FCC2A935A9E1575D08F@USDEN3EXMB003.na.aecomnet.com> Message-ID: <219D8EB618BE6940ACEC5F9D91B3BC7D0B06D3DB@bufex4.corp.ene.com> Hi Jay! If you need someone in Central Ohio I don't know of anyone.... We have a guy here based in Buffalo but he's not in Columbus... Greg From: Johnson, Jay [mailto:Jay.Johnson at aecom.com] Sent: Thursday, May 31, 2012 3:27 PM To: David at wwcols.com; Mailing list for Central Ohio Python User Group (COhPy) Cc: Coniglio, Gregory Subject: RE: [CentralOH] Way off topic, but any Access programmers available? David, GIS programmers usually are pretty good at those two items. I'm not sure who I would suggest in Columbus, but I know some good ones in Kentucky. Jay Greg, Any thoughts? Jay ________________________________ From: centraloh-bounces+jay.johnson=aecom.com at python.org [centraloh-bounces+jay.johnson=aecom.com at python.org] on behalf of David Chew [David at wwcols.com] Sent: Tuesday, May 29, 2012 10:18 AM To: centraloh at python.org Subject: [CentralOH] Way off topic, but any Access programmers available? Anyone work with Access & VBA? Realize not the place for that but you never know... I've got a job I'd like to hire out. Email me directly if interested. Click here to report this email as spam. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jep200404 at columbus.rr.com Mon Jun 4 16:28:17 2012 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Mon, 4 Jun 2012 10:28:17 -0400 Subject: [CentralOH] Django Management Command Memory Usage Message-ID: <20120604102817.2cf70b55.jep200404@columbus.rr.com> How can I reduce the memory usage in a Django management command? I have some Django code like follows in a management program: class Command(BaseCommand): ... def handle(self, *args, **options): for record in Places.objects.all(): if record.x and record.y: record.point = ( Point(float(record.x)/1000., float(record.y)/1000.)) else: record.point = None record.save() django.db.connection.close() In the settings.py file I have: DEBUG = False Places has millions of rows. top reveals that the program is using 18.6 Gigabytes of memory. How can I reduce that memory usage? Am I neglecting to close or release something? The only dox I'm finding about memory use related to query sets advise to use iterators instead of converting to a list. I'm already following that advice, but I'm not finding further guidance about memory use about record modification. Since DEBUG is False, I've already heeding the following. https://docs.djangoproject.com/en/dev/faq/models/#why-is-django-leaking-memory - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - I have found that the following might be nice, but doubt it addresses the memory issue. record.save(update_fields=['point']) From eric at intellovations.com Mon Jun 4 16:07:48 2012 From: eric at intellovations.com (Eric Floehr) Date: Mon, 4 Jun 2012 10:07:48 -0400 Subject: [CentralOH] The benefits of PyOhio Sponsorship In-Reply-To: References: Message-ID: Hi all, PyOhio has been blessed with many awesome sponsors, and 2012 is no exception. Many of the sponsors who helped make PyOhio such an awesome success last year have once again stepped up to the plate to sponsor again this year. PyOhio looks to be even bigger than the 200 Pythonistas who attended last year. Talk and tutorial proposals FAR exceed what was submitted last year. By all indications, PyOhio 2012 is going to be huge. I need your help. I would like to identify contacts (who are decision-makers) within companies who would benefit from sponsoring PyOhio. With possibly *four* concurrent tracks this year (up from three last year), our video production costs will be higher and we are especially looking for video sponsors. Video sponsors get their logo at the beginning of all the PyOhio videos at http://pyvideo.org/ -- it's the sponsorship that keeps on giving (in addition to all the regular benefits of sponsorship). PyOhio is a *free* to attend conference run by an *all-volunteer* staff who are dedicated to giving back to the community. We rely on our sponsors for the hard costs: venue rental, drinks and snacks, video recording and production, etc. We have submitted our application to become a 501(c) non-profit. Companies that might benefit from PyOhio sponsorship include: - Companies that sell tools, books, etc. to Python developers - Companies that hire Python developers - Companies that need Python developers now! - Companies that hire software developers in general - Local or regional companies that want greater name recognition - Companies that want to give back to the open source community Please contact me if you have any contacts or are a decision-maker in a company that would like to explore the benefits of sponsorship more fully. Thanks so much! Eric Floehr PyOhio 2012 -------------- next part -------------- An HTML attachment was scrubbed... URL: From kurtis.mullins at gmail.com Mon Jun 4 16:39:37 2012 From: kurtis.mullins at gmail.com (Kurtis Mullins) Date: Mon, 4 Jun 2012 10:39:37 -0400 Subject: [CentralOH] Django Management Command Memory Usage In-Reply-To: <20120604102817.2cf70b55.jep200404@columbus.rr.com> References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: Hey, It looks like you've got the fat trimmed off this one about as much as you can. I'd say filter out your results but even then you're still using every result. I'd recommend modifying this to run as raw SQL and I'm sure your memory usage will go down significantly. Pulling in this many records as Python Objects and conversely creating new Python objects on top of that (your Point objects) is most likely the cause of this memory usage. On Mon, Jun 4, 2012 at 10:28 AM, wrote: > How can I reduce the memory usage in a Django management command? > I have some Django code like follows in a management program: > > class Command(BaseCommand): > ... > def handle(self, *args, **options): > for record in Places.objects.all(): > if record.x and record.y: > record.point = ( > Point(float(record.x)/1000., > float(record.y)/1000.)) > else: > record.point = None > record.save() > django.db.connection.close() > > In the settings.py file I have: > > DEBUG = False > > Places has millions of rows. > top reveals that the program is using 18.6 Gigabytes of memory. > How can I reduce that memory usage? > Am I neglecting to close or release something? > > The only dox I'm finding about memory use related to query sets > advise to use iterators instead of converting to a list. > I'm already following that advice, but I'm not finding > further guidance about memory use about record modification. > > Since DEBUG is False, I've already heeding the following. > > > https://docs.djangoproject.com/en/dev/faq/models/#why-is-django-leaking-memory > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > I have found that the following might be nice, > but doubt it addresses the memory issue. > > record.save(update_fields=['point']) > > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > http://mail.python.org/mailman/listinfo/centraloh > -------------- next part -------------- An HTML attachment was scrubbed... URL: From godber at gmail.com Mon Jun 4 16:59:12 2012 From: godber at gmail.com (Austin Godber) Date: Mon, 4 Jun 2012 07:59:12 -0700 Subject: [CentralOH] Django Management Command Memory Usage In-Reply-To: <20120604102817.2cf70b55.jep200404@columbus.rr.com> References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: Hi, I've not had to solve this problem myself, yet. But if I had to there'd be two things I'd look into. Model pagination and perhaps using the query cache iterator method. #3 here: http://blog.davidziegler.net/post/548363214/some-common-django-orm-pitfalls First comment on: http://dayhacker.blogspot.com/2009/02/why-django-orm-sucks-it-takes-hell-lot.html I'd be interested to know your results if you try either of these. Austin On Mon, Jun 4, 2012 at 7:28 AM, wrote: > How can I reduce the memory usage in a Django management command? > I have some Django code like follows in a management program: > > class Command(BaseCommand): > ... > ? ?def handle(self, *args, **options): > ? ? ? ?for record in Places.objects.all(): > ? ? ? ? ? ?if record.x and record.y: > ? ? ? ? ? ? ? ?record.point = ( > ? ? ? ? ? ? ? ? ? ?Point(float(record.x)/1000., > ? ? ? ? ? ? ? ? ? ?float(record.y)/1000.)) > ? ? ? ? ? ?else: > ? ? ? ? ? ? ? ?record.point = None > ? ? ? ? ? ?record.save() > ? ? ? ?django.db.connection.close() > > In the settings.py file I have: > > DEBUG = False > > Places has millions of rows. > top reveals that the program is using 18.6 Gigabytes of memory. > How can I reduce that memory usage? > Am I neglecting to close or release something? > > The only dox I'm finding about memory use related to query sets > advise to use iterators instead of converting to a list. > I'm already following that advice, but I'm not finding > further guidance about memory use about record modification. > > Since DEBUG is False, I've already heeding the following. > > ? https://docs.djangoproject.com/en/dev/faq/models/#why-is-django-leaking-memory > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > I have found that the following might be nice, > but doubt it addresses the memory issue. > > ? ? ? ? ? ?record.save(update_fields=['point']) > > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > http://mail.python.org/mailman/listinfo/centraloh From eric at intellovations.com Mon Jun 4 17:08:56 2012 From: eric at intellovations.com (Eric Floehr) Date: Mon, 4 Jun 2012 11:08:56 -0400 Subject: [CentralOH] Django Management Command Memory Usage In-Reply-To: References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: Running as raw SQL will save memory (and possibly time) as you aren't loading each row from database to Python and back. The drawback would be you would be tied to whatever GIS implementation you are using (PostGIS supports a number). Here would be the equivalent PostGIS SQL to do this: update set point=ST_SetSRID(ST_Point(x/1000.0, y/1000.0), ) where x is not NULL and y is not NULL; The default in PostGIS is 4326, so that's likely what your point column will expect. A couple of notes: ST_Point, and most other PostGIS commands expect longitude,latitude form ... so in this case, x would be longitude and y latitude. Also, in your Python implementation, "if record.x and record.y" will fail when x or y is 0, which are valid, not just None. This has tripped me up in the past :-). So better would be "if record.x is not None and record.y is not None". Cheers, Eric On Mon, Jun 4, 2012 at 10:39 AM, Kurtis Mullins wrote: > Hey, > > It looks like you've got the fat trimmed off this one about as much as you > can. I'd say filter out your results but even then you're still using every > result. I'd recommend modifying this to run as raw SQL and I'm sure your > memory usage will go down significantly. Pulling in this many records as > Python Objects and conversely creating new Python objects on top of that > (your Point objects) is most likely the cause of this memory usage. > > > On Mon, Jun 4, 2012 at 10:28 AM, wrote: > >> How can I reduce the memory usage in a Django management command? >> I have some Django code like follows in a management program: >> >> class Command(BaseCommand): >> ... >> def handle(self, *args, **options): >> for record in Places.objects.all(): >> if record.x and record.y: >> record.point = ( >> Point(float(record.x)/1000., >> float(record.y)/1000.)) >> else: >> record.point = None >> record.save() >> django.db.connection.close() >> >> In the settings.py file I have: >> >> DEBUG = False >> >> Places has millions of rows. >> top reveals that the program is using 18.6 Gigabytes of memory. >> How can I reduce that memory usage? >> Am I neglecting to close or release something? >> >> The only dox I'm finding about memory use related to query sets >> advise to use iterators instead of converting to a list. >> I'm already following that advice, but I'm not finding >> further guidance about memory use about record modification. >> >> Since DEBUG is False, I've already heeding the following. >> >> >> https://docs.djangoproject.com/en/dev/faq/models/#why-is-django-leaking-memory >> >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> >> I have found that the following might be nice, >> but doubt it addresses the memory issue. >> >> record.save(update_fields=['point']) >> >> _______________________________________________ >> CentralOH mailing list >> CentralOH at python.org >> http://mail.python.org/mailman/listinfo/centraloh >> > > > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > http://mail.python.org/mailman/listinfo/centraloh > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jep200404 at columbus.rr.com Mon Jun 4 17:15:45 2012 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Mon, 4 Jun 2012 11:15:45 -0400 Subject: [CentralOH] None versus 0 Bug In-Reply-To: References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: <20120604111545.5defbf70.jep200404@columbus.rr.com> Hi Eric, On Mon, 4 Jun 2012 11:08:56 -0400, Eric Floehr wrote: > Also, in your Python implementation, "if record.x and record.y" will fail > when x or y is 0, which are valid, not just None. This has tripped me up > in the past :-). So better would be "if record.x is not None and record.y > is not None". Oops. Thanks for catching that. Jim From eric at intellovations.com Mon Jun 4 17:09:51 2012 From: eric at intellovations.com (Eric Floehr) Date: Mon, 4 Jun 2012 11:09:51 -0400 Subject: [CentralOH] Django Management Command Memory Usage In-Reply-To: References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: I should say GeoDjango supports a number of GIS implementations (not PostGIS, which is a specific implementation) :-) ... On Mon, Jun 4, 2012 at 11:08 AM, Eric Floehr wrote: > > Running as raw SQL will save memory (and possibly time) as you aren't > loading each row from database to Python and back. The drawback would be > you would be tied to whatever GIS implementation you are using (PostGIS > supports a number). Here would be the equivalent PostGIS SQL to do this: > > update set point=ST_SetSRID(ST_Point(x/1000.0, > y/1000.0), ) where x is not NULL and y is not NULL; > > The default in PostGIS is 4326, so that's likely what your point > column will expect. > > A couple of notes: > > ST_Point, and most other PostGIS commands expect longitude,latitude form > ... so in this case, x would be longitude and y latitude. > > Also, in your Python implementation, "if record.x and record.y" will fail > when x or y is 0, which are valid, not just None. This has tripped me up > in the past :-). So better would be "if record.x is not None and record.y > is not None". > > Cheers, > Eric > > > > > On Mon, Jun 4, 2012 at 10:39 AM, Kurtis Mullins wrote: > >> Hey, >> >> It looks like you've got the fat trimmed off this one about as much as >> you can. I'd say filter out your results but even then you're still using >> every result. I'd recommend modifying this to run as raw SQL and I'm sure >> your memory usage will go down significantly. Pulling in this many records >> as Python Objects and conversely creating new Python objects on top of that >> (your Point objects) is most likely the cause of this memory usage. >> >> >> On Mon, Jun 4, 2012 at 10:28 AM, wrote: >> >>> How can I reduce the memory usage in a Django management command? >>> I have some Django code like follows in a management program: >>> >>> class Command(BaseCommand): >>> ... >>> def handle(self, *args, **options): >>> for record in Places.objects.all(): >>> if record.x and record.y: >>> record.point = ( >>> Point(float(record.x)/1000., >>> float(record.y)/1000.)) >>> else: >>> record.point = None >>> record.save() >>> django.db.connection.close() >>> >>> In the settings.py file I have: >>> >>> DEBUG = False >>> >>> Places has millions of rows. >>> top reveals that the program is using 18.6 Gigabytes of memory. >>> How can I reduce that memory usage? >>> Am I neglecting to close or release something? >>> >>> The only dox I'm finding about memory use related to query sets >>> advise to use iterators instead of converting to a list. >>> I'm already following that advice, but I'm not finding >>> further guidance about memory use about record modification. >>> >>> Since DEBUG is False, I've already heeding the following. >>> >>> >>> https://docs.djangoproject.com/en/dev/faq/models/#why-is-django-leaking-memory >>> >>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >>> >>> I have found that the following might be nice, >>> but doubt it addresses the memory issue. >>> >>> record.save(update_fields=['point']) >>> >>> _______________________________________________ >>> CentralOH mailing list >>> CentralOH at python.org >>> http://mail.python.org/mailman/listinfo/centraloh >>> >> >> >> _______________________________________________ >> CentralOH mailing list >> CentralOH at python.org >> http://mail.python.org/mailman/listinfo/centraloh >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at microenh.com Mon Jun 4 17:44:01 2012 From: mark at microenh.com (Mr Mark E Erbaugh) Date: Mon, 4 Jun 2012 11:44:01 -0400 Subject: [CentralOH] Django Management Command Memory Usage In-Reply-To: <20120604102817.2cf70b55.jep200404@columbus.rr.com> References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: Jim, Does Places.objects.all() return an iterator or a sequence? Mark On Jun 4, 2012, at 10:28 AM, jep200404 at columbus.rr.com wrote: > def handle(self, *args, **options): > for record in Places.objects.all(): > if record.x and record.y: > record.point = ( > Point(float(record.x)/1000., > float(record.y)/1000.)) > else: > record.point = None > record.save() > django.db.connection.close() -------------- next part -------------- An HTML attachment was scrubbed... URL: From kurtis.mullins at gmail.com Mon Jun 4 17:50:08 2012 From: kurtis.mullins at gmail.com (Kurtis Mullins) Date: Mon, 4 Jun 2012 11:50:08 -0400 Subject: [CentralOH] Django Management Command Memory Usage In-Reply-To: References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: Places.objects.all() returns a Django Queryset object: https://docs.djangoproject.com/en/dev/ref/models/querysets/#queryset-api On Mon, Jun 4, 2012 at 11:44 AM, Mr Mark E Erbaugh wrote: > Jim, > > Does Places.objects.all() return an iterator or a sequence? > > Mark > > On Jun 4, 2012, at 10:28 AM, jep200404 at columbus.rr.com wrote: > > def handle(self, *args, **options): > for record in Places.objects.all(): > if record.x and record.y: > record.point = ( > Point(float(record.x)/1000., > float(record.y)/1000.)) > else: > record.point = None > record.save() > django.db.connection.close() > > > > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > http://mail.python.org/mailman/listinfo/centraloh > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jep200404 at columbus.rr.com Mon Jun 4 18:23:17 2012 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Mon, 4 Jun 2012 12:23:17 -0400 Subject: [CentralOH] update_fields keyword argument is new to Django 1.5 In-Reply-To: <20120604102817.2cf70b55.jep200404@columbus.rr.com> References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: <20120604122317.619d08c4.jep200404@columbus.rr.com> On Mon, 4 Jun 2012 10:28:17 -0400, jep200404 at columbus.rr.com wrote: > record.save(update_fields=['point']) By the way, the update_fields keyword argument is new in Django 1.5. https://docs.djangoproject.com/en/dev/releases/1.5/#support-for-saving-a-subset-of-model-s-fields From jep200404 at columbus.rr.com Mon Jun 4 18:53:41 2012 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Mon, 4 Jun 2012 12:53:41 -0400 Subject: [CentralOH] Places.objects.all() Seems to be An Iterator In-Reply-To: References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: <20120604125341.0d09df10.jep200404@columbus.rr.com> On Mon, 4 Jun 2012 11:44:01 -0400, Mr Mark E Erbaugh wrote: > Does Places.objects.all() return an iterator or a sequence? The dox that I've read[1] do not directly say, but suggest that .all() returns an iterator. See the Iteration and list() items in the list of ways that a QuerySet can be evaluated in the dox that I read. The memory usage has slowly increased as the program progressed. Last night, I noticed that it was a little over 10 GB and increasing. Considering how it had progressed, I expected it to use about 30 GB towards the end. Additionally, it currently seems stuck with memory usage stable at 18.6 GB and CPU usage around 1% (used to be high). [1] https://docs.djangoproject.com/en/1.4/ref/models/querysets/#when-querysets-are-evaluated [2] this also does not explicitly state whether what .all() returns (QuerySet) is an iterator or sequence. https://docs.djangoproject.com/en/dev/ref/models/querysets/#all From jep200404 at columbus.rr.com Mon Jun 4 21:34:56 2012 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Mon, 4 Jun 2012 15:34:56 -0400 Subject: [CentralOH] Django Management Command Memory Usage: Places.objects.all().iterator() Seems to be Solution In-Reply-To: References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: <20120604153456.6043cabe.jep200404@columbus.rr.com> On Mon, 4 Jun 2012 07:59:12 -0700, Austin Godber wrote: > #3 here: > http://blog.davidziegler.net/post/548363214/some-common-django-orm-pitfalls Thanks Austin. Places.objects.all().iterator() seems to have solved my problem. With the initial query, the memory use jumps up 3.5 GB, but does not increase further as the loop runs. It also seems faster, although I've not measured that. From jep200404 at columbus.rr.com Mon Jun 4 21:35:39 2012 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Mon, 4 Jun 2012 15:35:39 -0400 Subject: [CentralOH] Places.objects.all() Seems to be An Iterator; Then Again, Maybe Not In-Reply-To: <20120604125341.0d09df10.jep200404@columbus.rr.com> References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> <20120604125341.0d09df10.jep200404@columbus.rr.com> Message-ID: <20120604153539.2b0c7976.jep200404@columbus.rr.com> On Mon, 4 Jun 2012 12:53:41 -0400, jep200404 at columbus.rr.com wrote: > On Mon, 4 Jun 2012 11:44:01 -0400, Mr Mark E Erbaugh wrote: > > > Does Places.objects.all() return an iterator or a sequence? > > The dox that I've read[1] do not directly say, but suggest that > .all() returns an iterator. See the Iteration and list() items > in the list of ways that a QuerySet can be evaluated in the > dox that I read. Looking more, it's not clear. print >>self.stdout, repr(Places.objects.all()) seemed to indicate that foo was a list. Also, that one can append '.iterator()' suggests that .all() does not by itself return an iterator. So Places.objects.all() is looking more like a list (or more broadly, sequence) now. From jep200404 at columbus.rr.com Tue Jun 5 04:45:36 2012 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Mon, 4 Jun 2012 22:45:36 -0400 Subject: [CentralOH] Django Management Command Memory Usage: Chunkifying Worked In-Reply-To: <20120604153456.6043cabe.jep200404@columbus.rr.com> References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> <20120604153456.6043cabe.jep200404@columbus.rr.com> Message-ID: <20120604224536.690877e5.jep200404@columbus.rr.com> On Mon, 4 Jun 2012 15:34:56 -0400, jep200404 at columbus.rr.com wrote: > Places.objects.all().iterator() seems to have solved my problem. Well maybe not. The database was not getting updated. However, the following ugly code works and does so without gobbling too much memory. It seems to max out at 610 MB. def handle(self, *args, **options): chunk = 100000 min_id = Places.objects.all().aggregate(Min('id'))['id__min'] max_id = Places.objects.all().aggregate(Max('id'))['id__max'] for j in xrange(min_id, max_id + 1, chunk): for record in Places.objects.all().filter( id__gte=j).filter(id__lt=j + chunk): if record.x is not None and record.y is not None: record.point = ( Point(float(record.x)/1000., float(record.y)/1000.)) else: record.point = None record.save() django.db.connection.close() I'll play with Paginator per Nitin Hayaran another time. From eric at intellovations.com Tue Jun 5 20:12:54 2012 From: eric at intellovations.com (Eric Floehr) Date: Tue, 5 Jun 2012 14:12:54 -0400 Subject: [CentralOH] Fwd: SciPy 2012 in Austin, TX In-Reply-To: References: Message-ID: Information on SciPy, which will be in Austin, Texas from July 16-21... ---------- Forwarded message ---------- From: Majken Tranby All, As leaders of members of your local Python User groups I wanted to send you a message about SciPy 2012 in Austin Texas. >From July 16-21, this affordable conference will feature two days of tutorials including Introduction to Numpy, Ipython, HDF5, Parallel computing, scikits-learn, advanced Matplotlib, Pandas, and statsmodels.? Then we have a two day conference followed by some sprints. For more information please visit http://conference.scipy.org/scipy2012/. If you are willing to share this information with your respective groups that would be lovely! Best, Majken -- Majken E. Tranby Enthought Inc 515 Congress Avenue, Suite 2100 Austin, TX 78701 From jep200404 at columbus.rr.com Tue Jun 5 22:55:04 2012 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Tue, 5 Jun 2012 16:55:04 -0400 Subject: [CentralOH] Django Management Command Memory Usage In-Reply-To: References: <20120604102817.2cf70b55.jep200404@columbus.rr.com> Message-ID: <20120605165504.3f586b9d.jep200404@columbus.rr.com> On Mon, 4 Jun 2012 10:39:37 -0400, Kurtis Mullins wrote: > It looks like you've got the fat trimmed off this one about as much as you > can. I found a way. Using pre_save stuff elsewhere, I trimmed some more: def handle(self, *args, **options): for record in Places.objects.all(): record.save() django.db.connection.close() Yup. It's so trim now it looks like I forgot something. The ugly chunky version is: def handle(self, *args, **options): chunk = 100000 min_id = Places.objects.all().aggregate(Min('id'))['id__min'] max_id = Places.objects.all().aggregate(Max('id'))['id__max'] for j in xrange(min_id, max_id + 1, chunk): for record in Places.objects.all().filter( id__gte=j).filter(id__lt=j + chunk): record.save() django.db.connection.close() pre_save stuff is cool, very cool. From ETrogu at providepeople.com Tue Jun 12 19:48:04 2012 From: ETrogu at providepeople.com (Elisabetta Trogu) Date: Tue, 12 Jun 2012 18:48:04 +0100 Subject: [CentralOH] Job opportunity for Python developers Message-ID: <74705759C7D4224BADA0ECDA7DF285F201C8B593@PCSERVERDC.ProvideConsulting.local> Good evening I am looking for a number of Python developers to work for a fun company in the south west of England; I am looking for 3 Python developers at different levels: Junior ? 20-25K Mid-weight ? ?30-40K Senior ? up to 55K Any help is gonna be much appreciated Elisabetta Trogu Delivery Consultant DD: +44 (0)1908 524 852 Mob: +44 (0)7745 606 306 Web: www.providepeople.com Provide Consulting Limited is a registered company in England and Wales with company number 4411602 At the following location: 9 Shirwell Crescent, Furzton Lake, Milton Keynes, MK4 1GA Provide Consulting Limited is a VAT registered company with VAT number 763 7319 10 View a copy of our full terms and conditions here -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 2737 bytes Desc: image001.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 3514 bytes Desc: image002.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.png Type: image/png Size: 3546 bytes Desc: image003.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Python developer.docx Type: application/octet-stream Size: 14681 bytes Desc: Python developer.docx URL: From brian.costlow at gmail.com Wed Jun 20 02:04:27 2012 From: brian.costlow at gmail.com (Brian Costlow) Date: Tue, 19 Jun 2012 20:04:27 -0400 Subject: [CentralOH] PyOhio Registration Reminder Message-ID: About PyOhio PyOhio is one of the largest Python conferences in the midwest, and the largest entirely free Python conference anywhere. PyOhio 2012 will take place Saturday-Sunday, July 28-29, 2012 at the Ohio State University in Columbus, Ohio. A variety of activities are planned, including tutorials, scheduled talks, Lightning Talks, Open Spaces and Sprints. This year we doubled the number of talk proposals from last year, and will be trying to add a fourth track of talks and tutorials. If Registration is Free, Why Should I Register? The more people that register in advance, the better we can plan for space, snacks, drinks, etc. How Do I Register? Register Here: http://pyohio.org/register/ I'm Coming In From Out Of Town, Where Can I Stay? We're starting to pull together hotel information for you. There is no officially designated PyOhio hotel, but the nearest one to the Ohio Union is the Blackwell. http://www.theblackwell.com/ We've arranged a rate of $119.00 per night for PyOhio attendees (while the block lasts). To get the $119 rate, you need to reserve by phone (866-247-4003) and ask for the PyOhio rate. Here are some other hotels in the vicinity via Hipmunk: http://www.hipmunk.com/#!ohio+union+columbus+oh,Jul27.Jul29;1739+High+St+Columbus+OH+43210,Jul27.Jul29 How Can I Help? PyOhio is entirely volunteer run. If you would like to volunteer, check the "Contact Me, I'd Like To Volunteer" box when registering. Or go here and add your name: Volunteer Since the conference is free to attend, and volunteer-run, we rely on generous sponsors for things we have to pay for, like the venue, refreshments, printing, and videography. If you can sponsor the conference, or put us in touch with someone who might become a sponsor, respond back to this email with your contact information. Thanks! The PyOhio 2012 Organizing Committee Please feel free to forward this email to any and all interested parties. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kurtis.mullins at gmail.com Wed Jun 20 03:27:32 2012 From: kurtis.mullins at gmail.com (Kurtis Mullins) Date: Tue, 19 Jun 2012 21:27:32 -0400 Subject: [CentralOH] PyOhio Registration Reminder In-Reply-To: References: Message-ID: Brian, Do you think this would be an appropriate conference to bring a young child? I've never been before so I don't have any idea on the environment. Long talks might be tough to keep my son entertained and quiet :) Thanks! On Tue, Jun 19, 2012 at 8:04 PM, Brian Costlow wrote: > About PyOhio > > PyOhio is one of the largest Python conferences in the midwest, and the > largest entirely free Python conference anywhere. PyOhio 2012 will take > place Saturday-Sunday, July 28-29, 2012 at the Ohio State University in > Columbus, Ohio. A variety of activities are planned, including tutorials, > scheduled talks, Lightning Talks, Open Spaces and Sprints. > > This year we doubled the number of talk proposals from last year, and will > be trying to add a fourth track of talks and tutorials. > > If Registration is Free, Why Should I Register? > > The more people that register in advance, the better we can plan for > space, snacks, drinks, etc. > > How Do I Register? > > Register Here: http://pyohio.org/register/ > > I'm Coming In From Out Of Town, Where Can I Stay? > > We're starting to pull together hotel information for you. There is > no officially designated PyOhio hotel, but the nearest one to the Ohio > Union is the Blackwell. > > http://www.theblackwell.com/ > > We've arranged a rate of $119.00 per night for PyOhio attendees (while > the block lasts). To get the $119 rate, you need to reserve by phone > (866-247-4003) and ask for the PyOhio rate. > > Here are some other hotels in the vicinity via Hipmunk: > > > http://www.hipmunk.com/#!ohio+union+columbus+oh,Jul27.Jul29;1739+High+St+Columbus+OH+43210,Jul27.Jul29 > > How Can I Help? > > PyOhio is entirely volunteer run. If you would like to volunteer, check > the "Contact Me, I'd Like To Volunteer" box when registering. Or go here > and add your name: > > Volunteer > > Since the conference is free to attend, and volunteer-run, we rely on > generous sponsors for things we have to pay for, like the venue, > refreshments, printing, and videography. If you can sponsor the conference, > or put us in touch with someone who might become a sponsor, respond back to > this email with your contact information. > > Thanks! > > The PyOhio 2012 Organizing Committee > > Please feel free to forward this email to any and all interested parties. > > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > http://mail.python.org/mailman/listinfo/centraloh > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at intellovations.com Wed Jun 20 03:36:28 2012 From: eric at intellovations.com (Eric Floehr) Date: Tue, 19 Jun 2012 21:36:28 -0400 Subject: [CentralOH] PyOhio Registration Reminder In-Reply-To: References: Message-ID: How old is your child? Unlike Linuxfest there isn't much in the way of swag to keep a kid entertained, and little outside of the conference. There have been teens there, but generally they are teens interested in programming. -Eric On Tue, Jun 19, 2012 at 9:27 PM, Kurtis Mullins wrote: > Brian, > > Do you think this would be an appropriate conference to bring a young child? > I've never been before so I don't have any idea on the environment. Long > talks might be tough to keep my son entertained and quiet :) > > Thanks! > > On Tue, Jun 19, 2012 at 8:04 PM, Brian Costlow > wrote: >> >> About PyOhio >> >> PyOhio is one of the largest Python conferences in the midwest, and the >> largest entirely free Python conference anywhere. PyOhio 2012 will take >> place Saturday-Sunday, July 28-29, 2012 at the Ohio State University in >> Columbus, Ohio. A variety of activities are planned, including tutorials, >> scheduled talks, Lightning Talks, Open Spaces and Sprints. >> >> This year we doubled the number of talk proposals from last year, and will >> be trying to add a fourth track of talks and tutorials. >> >> If Registration is Free, Why Should I Register? >> >> The more people that register in advance, the better we can plan for >> space, snacks, drinks, etc. >> >> How Do I Register? >> >> Register Here: http://pyohio.org/register/ >> >> I'm Coming In From Out Of Town, Where Can I Stay? >> >> We're starting to pull together hotel information for you.? There is >> no officially designated PyOhio hotel, but the nearest one to the Ohio >> Union is the Blackwell. >> >> http://www.theblackwell.com/ >> >> We've arranged a rate of $119.00 per night for PyOhio attendees (while >> the block lasts).? To get the $119 rate, you need to reserve by phone >> (866-247-4003) and ask for the PyOhio rate. >> >> Here are some other hotels in the vicinity via Hipmunk: >> >> >> http://www.hipmunk.com/#!ohio+union+columbus+oh,Jul27.Jul29;1739+High+St+Columbus+OH+43210,Jul27.Jul29 >> >> How Can I Help? >> >> PyOhio is entirely volunteer run. If you would like to volunteer, check >> the "Contact Me, I'd Like To Volunteer" box when registering. Or go here and >> add your name: >> >> Volunteer >> >> Since the conference is free to attend, and volunteer-run, we rely on >> generous sponsors for things we have to pay for, like the venue, >> refreshments, printing, and videography. If you can sponsor the conference, >> or put us in touch with someone who might become a sponsor, respond back to >> this email with your contact information. >> >> Thanks! >> >> The PyOhio 2012 Organizing Committee >> >> Please feel free to forward this email to any and all interested parties. >> >> _______________________________________________ >> CentralOH mailing list >> CentralOH at python.org >> http://mail.python.org/mailman/listinfo/centraloh >> > > > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > http://mail.python.org/mailman/listinfo/centraloh >