From ryan.a.cox at gmail.com Thu Jul 8 18:23:56 2010 From: ryan.a.cox at gmail.com (Ryan Cox) Date: Thu, 8 Jul 2010 12:23:56 -0400 Subject: [TriZPUG] July TriHUG Meeting Message-ID: FYI: We have recently formed a Hadoop Users Group here in the Triangle. Our first meeting is scheduled for July 20th. Details in the link below. I plan on giving a talk on Processing Megadata with Hadoop and Python at the initial meeting. http://www.trihug.org/ Please tweet, blog and otherwise help spread the word! -ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From cbc at unc.edu Mon Jul 12 22:51:21 2010 From: cbc at unc.edu (Chris Calloway) Date: Mon, 12 Jul 2010 16:51:21 -0400 Subject: [TriZPUG] TriZPUG July 2010 Meeting: Accessibility with Python Message-ID: <4C3B8049.2080409@unc.edu> Next TriZPUG Meeting: Thursday July 22, 7pm at Carrboro Creative Coworking, 205 Lloyd Street, Suite 101, Carrboro, where parking is plentiful and free. Gary Bishop, Professor of Computer Science for the University of North Carolina at Chapel Hill develops free software for people with special needs. Gary will talk about the work he and his students are doing to enable people with disabilities. He will briefly cover several Python applications and libraries they have developed and then focus on how they are using Python as the back end of their UNC Open Web project. http://trizpug.org/Members/bac/july-10-mtg/ http://carrborocoworking.com/ http://wwwx.cs.unc.edu/~gb/ http://sites.google.com/site/uncopenweb/ As always, lightning talks of ten minutes or less are also welcome. Anything you've learned about Python, no matter how trivial, can be a lightning talk. Simply volunteer your lightning talk at the meeting. The after-meeting will be at one of the several nearby taverns, to be decided at the meeting. Thanks to Brad Crittenden for organizing this meeting. -- Sincerely, Chris Calloway office: 332 Chapman Hall phone: (919) 599-3530 mail: Campus Box #3300, UNC-CH, Chapel Hill, NC 27599 From ogmaciel at gnome.org Tue Jul 13 00:12:27 2010 From: ogmaciel at gnome.org (Og Maciel) Date: Mon, 12 Jul 2010 18:12:27 -0400 Subject: [TriZPUG] TriZPUG July 2010 Meeting: Accessibility with Python In-Reply-To: <4C3B8049.2080409@unc.edu> References: <4C3B8049.2080409@unc.edu> Message-ID: Awwww... I won't be around for this one... :/ Maybe someone can introduce me to Gary some time this week? I'd love to meet him and learn more about his work -- Og B. Maciel GNOME Foundation Board of Directors omaciel at foresightlinux.org ogmaciel at gnome.org ogmaciel at ubuntu.com GPG Keys: D5CFC202 http://www.ogmaciel.com (en_US) http://blog.ogmaciel.com (pt_BR) From cbc at unc.edu Wed Jul 21 18:26:47 2010 From: cbc at unc.edu (Chris Calloway) Date: Wed, 21 Jul 2010 12:26:47 -0400 Subject: [TriZPUG] TriZPUG July 2010 Meeting: Accessibility with Python In-Reply-To: <4C3B8049.2080409@unc.edu> References: <4C3B8049.2080409@unc.edu> Message-ID: <4C471FC7.8040807@unc.edu> Reminder: this meeting is tomorrow, Thursday, June 22: On 7/12/2010 4:51 PM, Chris Calloway wrote: > Next TriZPUG Meeting: Thursday July 22, 7pm at Carrboro Creative > Coworking, 205 Lloyd Street, Suite 101, Carrboro, where parking is > plentiful and free. Gary Bishop, Professor of Computer Science for the > University of North Carolina at Chapel Hill develops free software for > people with special needs. Gary will talk about the work he and his > students are doing to enable people with disabilities. He will briefly > cover several Python applications and libraries they have developed and > then focus on how they are using Python as the back end of their UNC > Open Web project. > > http://trizpug.org/Members/bac/july-10-mtg/ > http://carrborocoworking.com/ > http://wwwx.cs.unc.edu/~gb/ > http://sites.google.com/site/uncopenweb/ > > As always, lightning talks of ten minutes or less are also welcome. > Anything you've learned about Python, no matter how trivial, can be a > lightning talk. Simply volunteer your lightning talk at the meeting. The > after-meeting will be at one of the several nearby taverns, to be > decided at the meeting. > > Thanks to Brad Crittenden for organizing this meeting. > -- Sincerely, Chris Calloway office: 332 Chapman Hall phone: (919) 599-3530 mail: Campus Box #3300, UNC-CH, Chapel Hill, NC 27599 From dragonstrider at gmail.com Fri Jul 30 08:04:07 2010 From: dragonstrider at gmail.com (Joseph Tate) Date: Fri, 30 Jul 2010 02:04:07 -0400 Subject: [TriZPUG] Slow networking to localhost Message-ID: I'm running into a weird networking issue with one of my web apps. We have a baseline install and a customized install. The customized install takes about 5 times longer to load a page than the base install does when running from the same virtualenv on the same hardware and using the same browser to the app running on localhost. According to cProfiler, for the slow install, 91.56% of the time is being spent in "select.select". Strace shows that the select method is timing out multiple times, as if the traffic between the browser and my app is stalling. Any ideas on what to check next? -- Joseph Tate Personal e-mail: jtate AT dragonstrider DOT com Web: http://www.dragonstrider.com From josh_johnson at unc.edu Fri Jul 30 14:10:53 2010 From: josh_johnson at unc.edu (Josh Johnson) Date: Fri, 30 Jul 2010 08:10:53 -0400 Subject: [TriZPUG] Slow networking to localhost In-Reply-To: References: Message-ID: <005101cb2fe0$4211be70$c6353b50$@edu> It's hard to tell what might be going on without any real information about what the web app is, what it does, what modules it uses, any frameworks, etc. Are you talking about installing the two versions of the web app (I'm assuming it's an egg? If not how is it installed?) in the same virtualenv, or two different virtualenvs that you think are set up the same way? Also, when you say "customized install" what does that mean? Is the installation customized, or is the web app customized? More information would really help, but my approach to an issue like this is to focus on the differences. A simple diff of the code might make the problem stand out. The cProfiler result may be a red herring, but you may want to check for any known bugs in python or your OS relating to it, given the nature of the select module. JJ -----Original Message----- From: trizpug-bounces+josh_johnson=unc.edu at python.org [mailto:trizpug-bounces+josh_johnson=unc.edu at python.org] On Behalf Of Joseph Tate Sent: Friday, July 30, 2010 2:04 AM To: Triangle (North Carolina) Zope and Python Users Group Subject: [TriZPUG] Slow networking to localhost I'm running into a weird networking issue with one of my web apps. We have a baseline install and a customized install. The customized install takes about 5 times longer to load a page than the base install does when running from the same virtualenv on the same hardware and using the same browser to the app running on localhost. According to cProfiler, for the slow install, 91.56% of the time is being spent in "select.select". Strace shows that the select method is timing out multiple times, as if the traffic between the browser and my app is stalling. Any ideas on what to check next? -- Joseph Tate Personal e-mail: jtate AT dragonstrider DOT com Web: http://www.dragonstrider.com _______________________________________________ TriZPUG mailing list TriZPUG at python.org http://mail.python.org/mailman/listinfo/trizpug http://trizpug.org is the Triangle Zope and Python Users Group From jim at ibang.com Fri Jul 30 15:23:12 2010 From: jim at ibang.com (Jim Allman) Date: Fri, 30 Jul 2010 09:23:12 -0400 Subject: [TriZPUG] Slow networking to localhost In-Reply-To: References: Message-ID: On Jul 30, 2010, at 2:04 AM, Joseph Tate wrote: > I'm running into a weird networking issue with one of my web apps. .... > According to cProfiler, for the slow install, 91.56% of the time is > being spent in "select.select". Strace shows that the select method > is timing out multiple times, as if the traffic between the browser > and my app is stalling. Joseph, Are both versions of the app using the same data? If 'select.select' is an operation that involves querying a database, it's possible that your "custom" web app is timing out before the response is complete. This might lead to retries as your describing, until you basically "get lucky" and make it across the finish line in time. (I see this a lot in PHP websites, since its default timeout on web requests is very short.) =jimA= From dragonstrider at gmail.com Fri Jul 30 18:44:12 2010 From: dragonstrider at gmail.com (Joseph Tate) Date: Fri, 30 Jul 2010 12:44:12 -0400 Subject: [TriZPUG] Slow networking to localhost In-Reply-To: <005101cb2fe0$4211be70$c6353b50$@edu> References: <005101cb2fe0$4211be70$c6353b50$@edu> Message-ID: It's a wsgi app using pylons. The app, and the customization module are both installed in a single virtenv, but the customized version has a different config file, slightly different routing and different data. The main app is made up of about 10 namespaced modules, the customized version isn't monkey patching anything, but rather adds on an additional module, has its own middleware setup and so forth. I diffed the config and middleware and I didn't see anything that made sense. All the packages are installed via setup.py develop. What I profiled was the paster process, so you may be right on select being a red herring (multiple threads waiting on a socket, etc.) The really confusing part though is it's the request round trip that's slower, not (just) the data lookups. Each static file is served 5 times slower in the customized config than was happening with the base config. As far as I can tell, the middleware and static file serving is configured the same for both apps. The only thing I can think of would be if we're somehow double stacking the middleware. On Fri, Jul 30, 2010 at 8:10 AM, Josh Johnson wrote: > It's hard to tell what might be going on without any real information about > what the web app is, what it does, what modules it uses, any frameworks, > etc. > > Are you talking about installing the two versions of the web app (I'm > assuming it's an egg? If not how is it installed?) in the same virtualenv, > or two different virtualenvs that you think are set up the same way? > > Also, when you say "customized install" what does that mean? Is the > installation customized, or is the web app customized? > > More information would really help, but my approach to an issue like this is > to focus on the differences. A simple diff of the code might make the > problem stand out. The cProfiler result may be a red herring, but you may > want to check for any known bugs in python or your OS relating to it, given > the nature of the select module. > > JJ > > -----Original Message----- > From: trizpug-bounces+josh_johnson=unc.edu at python.org > [mailto:trizpug-bounces+josh_johnson=unc.edu at python.org] On Behalf Of Joseph > Tate > Sent: Friday, July 30, 2010 2:04 AM > To: Triangle (North Carolina) Zope and Python Users Group > Subject: [TriZPUG] Slow networking to localhost > > I'm running into a weird networking issue with one of my web apps. ?We > have a baseline install and a customized install. ?The customized > install takes about 5 times longer to load a page than the base > install does when running from the same virtualenv on the same > hardware and using the same browser to the app running on localhost. > According to cProfiler, for the slow install, 91.56% of the time is > being spent in "select.select". ?Strace shows that the select method > is timing out multiple times, as if the traffic between the browser > and my app is stalling. ?Any ideas on what to check next? > > -- > Joseph Tate > Personal e-mail: jtate AT dragonstrider DOT com > Web: http://www.dragonstrider.com > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > http://mail.python.org/mailman/listinfo/trizpug > http://trizpug.org is the Triangle Zope and Python Users Group > > > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > http://mail.python.org/mailman/listinfo/trizpug > http://trizpug.org is the Triangle Zope and Python Users Group > -- Joseph Tate Personal e-mail: jtate AT dragonstrider DOT com Web: http://www.dragonstrider.com From dragonstrider at gmail.com Fri Jul 30 18:45:46 2010 From: dragonstrider at gmail.com (Joseph Tate) Date: Fri, 30 Jul 2010 12:45:46 -0400 Subject: [TriZPUG] Slow networking to localhost In-Reply-To: References: Message-ID: On Fri, Jul 30, 2010 at 9:23 AM, Jim Allman wrote: > Are both versions of the app using the same data? If 'select.select' is an operation that involves querying a database, it's possible that your "custom" web app is timing out before the response is complete. This might lead to retries as your describing, until you basically "get lucky" and make it across the finish line in time. > > (I see this a lot in PHP websites, since its default timeout on web requests is very short.) > No, using different data, but the slowness is evident in the static file serving as well as the data lookups, so I don't think it's db bottlenecks. -- Joseph Tate Personal e-mail: jtate AT dragonstrider DOT com Web: http://www.dragonstrider.com From josh_johnson at unc.edu Fri Jul 30 18:52:03 2010 From: josh_johnson at unc.edu (Josh Johnson) Date: Fri, 30 Jul 2010 12:52:03 -0400 Subject: [TriZPUG] Slow networking to localhost In-Reply-To: References: <005101cb2fe0$4211be70$c6353b50$@edu> Message-ID: <009e01cb3007$89b6bd30$9d243790$@edu> If you could post the config files, and routing setup that would help a lot. Maybe you're just missing something simple in the config/routing that somebody else might catch. JJ -----Original Message----- From: trizpug-bounces+josh_johnson=unc.edu at python.org [mailto:trizpug-bounces+josh_johnson=unc.edu at python.org] On Behalf Of Joseph Tate Sent: Friday, July 30, 2010 12:44 PM To: Triangle (North Carolina) Zope and Python Users Group Subject: Re: [TriZPUG] Slow networking to localhost It's a wsgi app using pylons. The app, and the customization module are both installed in a single virtenv, but the customized version has a different config file, slightly different routing and different data. The main app is made up of about 10 namespaced modules, the customized version isn't monkey patching anything, but rather adds on an additional module, has its own middleware setup and so forth. I diffed the config and middleware and I didn't see anything that made sense. All the packages are installed via setup.py develop. What I profiled was the paster process, so you may be right on select being a red herring (multiple threads waiting on a socket, etc.) The really confusing part though is it's the request round trip that's slower, not (just) the data lookups. Each static file is served 5 times slower in the customized config than was happening with the base config. As far as I can tell, the middleware and static file serving is configured the same for both apps. The only thing I can think of would be if we're somehow double stacking the middleware. On Fri, Jul 30, 2010 at 8:10 AM, Josh Johnson wrote: > It's hard to tell what might be going on without any real information about > what the web app is, what it does, what modules it uses, any frameworks, > etc. > > Are you talking about installing the two versions of the web app (I'm > assuming it's an egg? If not how is it installed?) in the same virtualenv, > or two different virtualenvs that you think are set up the same way? > > Also, when you say "customized install" what does that mean? Is the > installation customized, or is the web app customized? > > More information would really help, but my approach to an issue like this is > to focus on the differences. A simple diff of the code might make the > problem stand out. The cProfiler result may be a red herring, but you may > want to check for any known bugs in python or your OS relating to it, given > the nature of the select module. > > JJ > > -----Original Message----- > From: trizpug-bounces+josh_johnson=unc.edu at python.org > [mailto:trizpug-bounces+josh_johnson=unc.edu at python.org] On Behalf Of Joseph > Tate > Sent: Friday, July 30, 2010 2:04 AM > To: Triangle (North Carolina) Zope and Python Users Group > Subject: [TriZPUG] Slow networking to localhost > > I'm running into a weird networking issue with one of my web apps. ?We > have a baseline install and a customized install. ?The customized > install takes about 5 times longer to load a page than the base > install does when running from the same virtualenv on the same > hardware and using the same browser to the app running on localhost. > According to cProfiler, for the slow install, 91.56% of the time is > being spent in "select.select". ?Strace shows that the select method > is timing out multiple times, as if the traffic between the browser > and my app is stalling. ?Any ideas on what to check next? > > -- > Joseph Tate > Personal e-mail: jtate AT dragonstrider DOT com > Web: http://www.dragonstrider.com > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > http://mail.python.org/mailman/listinfo/trizpug > http://trizpug.org is the Triangle Zope and Python Users Group > > > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > http://mail.python.org/mailman/listinfo/trizpug > http://trizpug.org is the Triangle Zope and Python Users Group > -- Joseph Tate Personal e-mail: jtate AT dragonstrider DOT com Web: http://www.dragonstrider.com _______________________________________________ TriZPUG mailing list TriZPUG at python.org http://mail.python.org/mailman/listinfo/trizpug http://trizpug.org is the Triangle Zope and Python Users Group