From cbc at unc.edu Tue Jun 6 09:38:49 2017 From: cbc at unc.edu (Calloway, Chris) Date: Tue, 6 Jun 2017 13:38:49 +0000 Subject: [TriPython] Reminder: Raleigh Project Night Tonight Message-ID: <5FB411EF-99C6-4B59-94C1-1946360D427C@unc.edu> I?m looking forward to project night tonight at WebAssign on the NCSU Centennial Campus. Pizza will be provided. http://tripython.org/Members/sgambino/june-17-rpn/ When: Tuesday, June 6, 6-9pm Where: WebAssign NCSU Centennial Campus 1791 Varsity Drive, Suite 200, Raleigh What: Raleigh Project Night meets on first Tuesdays. Have a project you want to show off, share, seek help with, or just get some work done surrounded by like-minded Python lovers? Join us for our monthly project night and do just that! Don't have something to work on? Just need some help with Python? Show up and enjoy the energy, sprint on an open source project, find something interesting to contribute to or be inspired by! The setting is informal and there is no schedule, so don't worry if you show up past the start time. Whether you are a Python newbie needing help or have an open source project you want to share, come hang out and hack. Plenty of free after hours parking is available in the upper level of the deck behind WebAssign (turn through the median just before the intersection of Varsity and Main Campus Drives). If the door is locked, call the number posted on the door. Bring your laptop. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 -------------- next part -------------- I'm looking forward to project night tonight at WebAssign on the NCSU Centennial Campus. Pizza will be provided. http://tripython.org/Members/sgambino/june-17-rpn/ When: Tuesday, June 6, 6-9pm Where: WebAssign NCSU Centennial Campus 1791 Varsity Drive, Suite 200, Raleigh What: Raleigh Project Night meets on first Tuesdays. Have a project you want to show off, share, seek help with, or just get some work done surrounded by like-minded Python lovers? Join us for our monthly project night and do just that! Don't have something to work on? Just need some help with Python? Show up and enjoy the energy, sprint on an open source project, find something interesting to contribute to or be inspired by! The setting is informal and there is no schedule, so don't worry if you show up past the start time. Whether you are a Python newbie needing help or have an open source project you want to share, come hang out and hack. Plenty of free after hours parking is available in the upper level of the deck behind WebAssign (turn through the median just before the intersection of Varsity and Main Campus Drives). If the door is locked, call the number posted on the door. Bring your laptop. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 From ken at mack-z.com Tue Jun 6 11:43:56 2017 From: ken at mack-z.com (Ken MacKenzie) Date: Tue, 6 Jun 2017 11:43:56 -0400 Subject: [TriPython] Places to look at performance tuning Message-ID: So I am in the demo and test phase of an early ReSTful API for reporting. Currently a wider scale report set request hits the following marks: TTFB: ~20s Record Count: ~92k Download Size: 15.8MB Details: Web Server: NGINX Python App Server: Gunicorn Web Framework: Falcon Python version: 3.5 (in a venv) DB: MS SQL Server Express using SQL Alchemy + pyodbc Webserver OS: CentOS 7 Gunicorn is setup with 4 workers, on a private port, nginx does a proxy pass to the port DB Details, the table in question has a total of about 8 million rows. Sample query execution within SQL Server Mgmt Studio is ~7s So my question is which of the following would be a better target to improve performance, or do I need to as my performance should be considered good enough. I mean the server in this case is a surplus dual core desktop right now. add gzip compression to nginx for proxys switch gunicorn to use a unix socket instead of a tcp port consider leaner SQL and JSON marshaling requests instead of ORM's and dictionary bundles. Appreciate and advice or suggestions. Thank you. -------------- next part -------------- So I am in the demo and test phase of an early ReSTful API for reporting. Currently a wider scale report set request hits the following marks: TTFB: ~20s Record Count: ~92k Download Size: 15.8MB Details: Web Server: NGINX Python App Server: Gunicorn Web Framework: Falcon Python version: 3.5 (in a venv) DB: MS SQL Server Express using SQL Alchemy + pyodbc Webserver OS: CentOS 7 Gunicorn is setup with 4 workers, on a private port, nginx does a proxy pass to the port DB Details, the table in question has a total of about 8 million rows.** Sample query execution within SQL Server Mgmt Studio is ~7s So my question is which of the following would be a better target to improve performance, or do I need to as my performance should be considered good enough.** I mean the server in this case is a surplus dual core desktop right now. add gzip compression to nginx for proxys switch gunicorn to use a unix socket instead of a tcp port consider leaner SQL and JSON marshaling requests instead of ORM's and dictionary bundles. Appreciate and advice or suggestions.** Thank you. From george.gergues at gmail.com Tue Jun 6 16:18:11 2017 From: george.gergues at gmail.com (George Gergues) Date: Tue, 6 Jun 2017 16:18:11 -0400 Subject: [TriPython] Places to look at performance tuning In-Reply-To: References: Message-ID: For SQL table add at least one index. it will improve table scans. On Jun 6, 2017 11:44, "Ken MacKenzie" wrote: > So I am in the demo and test phase of an early ReSTful API for > reporting. > Currently a wider scale report set request hits the following marks: > TTFB: ~20s > Record Count: ~92k > Download Size: 15.8MB > Details: > Web Server: NGINX > Python App Server: Gunicorn > Web Framework: Falcon > Python version: 3.5 (in a venv) > DB: MS SQL Server Express using SQL Alchemy + pyodbc > Webserver OS: CentOS 7 > Gunicorn is setup with 4 workers, on a private port, nginx does a proxy > pass to the port > DB Details, the table in question has a total of about 8 million rows.** > Sample query execution within SQL Server Mgmt Studio is ~7s > So my question is which of the following would be a better target to > improve performance, or do I need to as my performance should be > considered good enough.** I mean the server in this case is a surplus > dual > core desktop right now. > add gzip compression to nginx for proxys > switch gunicorn to use a unix socket instead of a tcp port > consider leaner SQL and JSON marshaling requests instead of ORM's and > dictionary bundles. > Appreciate and advice or suggestions.** Thank you. > > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > https://mail.python.org/mailman/listinfo/trizpug > http://tripython.org is the Triangle Python Users Group > > -------------- next part -------------- For SQL table **add at least one index. it will improve table scans. On Jun 6, 2017 11:44, "Ken MacKenzie" <[1]ken at mack-z.com> wrote: ** **So I am in the demo and test phase of an early ReSTful API for reporting. ** **Currently a wider scale report set request hits the following marks: ** **TTFB: ~20s ** **Record Count: ~92k ** **Download Size: 15.8MB ** **Details: ** **Web Server: NGINX ** **Python App Server: Gunicorn ** **Web Framework: Falcon ** **Python version: 3.5 (in a venv) ** **DB: MS SQL Server Express using SQL Alchemy + pyodbc ** **Webserver OS: CentOS 7 ** **Gunicorn is setup with 4 workers, on a private port, nginx does a proxy ** **pass to the port ** **DB Details, the table in question has a total of about 8 million rows.** ** **Sample query execution within SQL Server Mgmt Studio is ~7s ** **So my question is which of the following would be a better target to ** **improve performance, or do I need to as my performance should be ** **considered good enough.** I mean the server in this case is a surplus dual ** **core desktop right now. ** **add gzip compression to nginx for proxys ** **switch gunicorn to use a unix socket instead of a tcp port ** **consider leaner SQL and JSON marshaling requests instead of ORM's and ** **dictionary bundles. ** **Appreciate and advice or suggestions.** Thank you. _______________________________________________ TriZPUG mailing list [2]TriZPUG at python.org [3]https://mail.python.org/mailman/listinfo/trizpug [4]http://tripython.org is the Triangle Python Users Group References Visible links 1. mailto:ken at mack-z.com 2. mailto:TriZPUG at python.org 3. https://mail.python.org/mailman/listinfo/trizpug 4. http://tripython.org/ From ken at mack-z.com Tue Jun 6 16:28:18 2017 From: ken at mack-z.com (Ken MacKenzie) Date: Tue, 6 Jun 2017 16:28:18 -0400 Subject: [TriPython] Places to look at performance tuning In-Reply-To: References: Message-ID: At present the SQL table has about 8 fields that make up the primary key, that PK creates an index and is the only index at present. I don't think sql selection is my bottle neck. here is why. basically my route is like this: /type/fiscal_year/fiscal_period/entity entity is the only one that is "optional" When an entity is specified I get a return in a second or two. Granted a much smaller record set. When entity is omitted I get about 50x the result count and that is where I get to a 20 second return time. Those results at first made me think the problem was io bound on the router till I saw that the first return byte was so behind. Am I misunderstanding your suggestion that additional indexes would improve the speed at which the DB returns the data, because I am interpreting your suggestion as one to improve the selection speed. On Tue, Jun 6, 2017 at 4:18 PM, George Gergues wrote: > For SQL table **add at least one index. it will improve table scans. > On Jun 6, 2017 11:44, "Ken MacKenzie" <[1]ken at mack-z.com> wrote: > > ** **So I am in the demo and test phase of an early ReSTful API for > reporting. > ** **Currently a wider scale report set request hits the following > marks: > ** **TTFB: ~20s > ** **Record Count: ~92k > ** **Download Size: 15.8MB > ** **Details: > ** **Web Server: NGINX > ** **Python App Server: Gunicorn > ** **Web Framework: Falcon > ** **Python version: 3.5 (in a venv) > ** **DB: MS SQL Server Express using SQL Alchemy + pyodbc > ** **Webserver OS: CentOS 7 > ** **Gunicorn is setup with 4 workers, on a private port, nginx does a > proxy > ** **pass to the port > ** **DB Details, the table in question has a total of about 8 million > rows.** > ** **Sample query execution within SQL Server Mgmt Studio is ~7s > ** **So my question is which of the following would be a better target > to > ** **improve performance, or do I need to as my performance should be > ** **considered good enough.** I mean the server in this case is a > surplus dual > ** **core desktop right now. > ** **add gzip compression to nginx for proxys > ** **switch gunicorn to use a unix socket instead of a tcp port > ** **consider leaner SQL and JSON marshaling requests instead of ORM's > and > ** **dictionary bundles. > ** **Appreciate and advice or suggestions.** Thank you. > > _______________________________________________ > TriZPUG mailing list > [2]TriZPUG at python.org > [3]https://mail.python.org/mailman/listinfo/trizpug > [4]http://tripython.org is the Triangle Python Users Group > > -------------- next part -------------- At present the SQL table has about 8 fields that make up the primary key, that PK creates an index and is the only index at present. I don't think sql selection is my bottle neck. **here is why. basically my route is like this: /type/fiscal_year/fiscal_period/entity entity is the only one that is "optional" When an entity is specified I get a return in a second or two.** Granted a much smaller record set. When entity is omitted I get about 50x the result count and that is where I get to a 20 second return time. Those results at first made me think the problem was io bound on the router till I saw that the first return byte was so behind. Am I misunderstanding your suggestion that additional indexes would improve the speed at which the DB returns the data, because I am interpreting your suggestion as one to improve the selection speed. On Tue, Jun 6, 2017 at 4:18 PM, George Gergues <[1]george.gergues at gmail.com> wrote: ** **For SQL table **add at least one index. it will improve table scans. ** **On Jun 6, 2017 11:44, "Ken MacKenzie" <[1][2]ken at mack-z.com> wrote: ** ** **** **So I am in the demo and test phase of an early ReSTful API for ** ** **reporting. ** ** **** **Currently a wider scale report set request hits the following ** ** **marks: ** ** **** **TTFB: ~20s ** ** **** **Record Count: ~92k ** ** **** **Download Size: 15.8MB ** ** **** **Details: ** ** **** **Web Server: NGINX ** ** **** **Python App Server: Gunicorn ** ** **** **Web Framework: Falcon ** ** **** **Python version: 3.5 (in a venv) ** ** **** **DB: MS SQL Server Express using SQL Alchemy + pyodbc ** ** **** **Webserver OS: CentOS 7 ** ** **** **Gunicorn is setup with 4 workers, on a private port, nginx does a ** ** **proxy ** ** **** **pass to the port ** ** **** **DB Details, the table in question has a total of about 8 million ** ** **rows.** ** ** **** **Sample query execution within SQL Server Mgmt Studio is ~7s ** ** **** **So my question is which of the following would be a better target ** ** **to ** ** **** **improve performance, or do I need to as my performance should be ** ** **** **considered good enough.** I mean the server in this case is a ** ** **surplus dual ** ** **** **core desktop right now. ** ** **** **add gzip compression to nginx for proxys ** ** **** **switch gunicorn to use a unix socket instead of a tcp port ** ** **** **consider leaner SQL and JSON marshaling requests instead of ORM's ** ** **and ** ** **** **dictionary bundles. ** ** **** **Appreciate and advice or suggestions.** Thank you. ** ** **_______________________________________________ ** ** **TriZPUG mailing list ** ** **[2][3]TriZPUG at python.org ** ** **[3][4]https://mail.python.org/mailman/listinfo/trizpug ** ** **[4][5]http://tripython.org is the Triangle Python Users Group References Visible links 1. mailto:george.gergues at gmail.com 2. mailto:ken at mack-z.com 3. mailto:TriZPUG at python.org 4. https://mail.python.org/mailman/listinfo/trizpug 5. http://tripython.org/ From jwhisnant at gmail.com Tue Jun 6 16:44:04 2017 From: jwhisnant at gmail.com (James Whisnant) Date: Tue, 6 Jun 2017 16:44:04 -0400 Subject: [TriPython] Places to look at performance tuning In-Reply-To: References: Message-ID: I recommend profiling your code to see exactly where it is slow. It may or may not be what you expect. https://pypi.python.org/pypi/profilehooks is an easy to use decorate your functions to profile them. It does sound like some of your database queries are slow. This may help you pinpoint where and why. Are you using the SQLAlchemy ORM or Core? The ORM can have a large overhead (in some cases) as compared to the SQLAlchemy Core. On Tue, Jun 6, 2017 at 4:28 PM, Ken MacKenzie wrote: > At present the SQL table has about 8 fields that make up the primary > key, > that PK creates an index and is the only index at present. > I don't think sql selection is my bottle neck. **here is why. > basically my route is like this: > /type/fiscal_year/fiscal_period/entity > entity is the only one that is "optional" > When an entity is specified I get a return in a second or two.** > Granted a > much smaller record set. > When entity is omitted I get about 50x the result count and that is > where > I get to a 20 second return time. > Those results at first made me think the problem was io bound on the > router till I saw that the first return byte was so behind. > Am I misunderstanding your suggestion that additional indexes would > improve the speed at which the DB returns the data, because I am > interpreting your suggestion as one to improve the selection speed. > On Tue, Jun 6, 2017 at 4:18 PM, George Gergues > <[1]george.gergues at gmail.com> wrote: > > ** **For SQL table **add at least one index. it will improve table > scans. > ** **On Jun 6, 2017 11:44, "Ken MacKenzie" <[1][2]ken at mack-z.com> > wrote: > > ** ** **** **So I am in the demo and test phase of an early ReSTful > API > for > ** ** **reporting. > ** ** **** **Currently a wider scale report set request hits the > following > ** ** **marks: > ** ** **** **TTFB: ~20s > ** ** **** **Record Count: ~92k > ** ** **** **Download Size: 15.8MB > ** ** **** **Details: > ** ** **** **Web Server: NGINX > ** ** **** **Python App Server: Gunicorn > ** ** **** **Web Framework: Falcon > ** ** **** **Python version: 3.5 (in a venv) > ** ** **** **DB: MS SQL Server Express using SQL Alchemy + pyodbc > ** ** **** **Webserver OS: CentOS 7 > ** ** **** **Gunicorn is setup with 4 workers, on a private port, > nginx > does a > ** ** **proxy > ** ** **** **pass to the port > ** ** **** **DB Details, the table in question has a total of about 8 > million > ** ** **rows.** > ** ** **** **Sample query execution within SQL Server Mgmt Studio is > ~7s > ** ** **** **So my question is which of the following would be a > better > target > ** ** **to > ** ** **** **improve performance, or do I need to as my performance > should be > ** ** **** **considered good enough.** I mean the server in this case > is > a > ** ** **surplus dual > ** ** **** **core desktop right now. > ** ** **** **add gzip compression to nginx for proxys > ** ** **** **switch gunicorn to use a unix socket instead of a tcp > port > ** ** **** **consider leaner SQL and JSON marshaling requests instead > of > ORM's > ** ** **and > ** ** **** **dictionary bundles. > ** ** **** **Appreciate and advice or suggestions.** Thank you. > > ** ** **_______________________________________________ > ** ** **TriZPUG mailing list > ** ** **[2][3]TriZPUG at python.org > ** ** **[3][4]https://mail.python.org/mailman/listinfo/trizpug > ** ** **[4][5]http://tripython.org is the Triangle Python Users Group > > References > > Visible links > 1. mailto:george.gergues at gmail.com > 2. mailto:ken at mack-z.com > 3. mailto:TriZPUG at python.org > 4. https://mail.python.org/mailman/listinfo/trizpug > 5. http://tripython.org/ > > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > https://mail.python.org/mailman/listinfo/trizpug > http://tripython.org is the Triangle Python Users Group > > -------------- next part -------------- I recommend profiling your code to see exactly where it is slow. It may or may not be what you expect. [1]https://pypi.python.org/pypi/profilehooks is an easy to use decorate your functions to profile them. It does sound like some of your database queries are slow. This may help you pinpoint where and why. Are you using the SQLAlchemy ORM or Core? The ORM can have a large overhead (in some cases) as compared to the SQLAlchemy Core. On Tue, Jun 6, 2017 at 4:28 PM, Ken MacKenzie <[2]ken at mack-z.com> wrote: ** **At present the SQL table has about 8 fields that make up the primary key, ** **that PK creates an index and is the only index at present. ** **I don't think sql selection is my bottle neck. **here is why. ** **basically my route is like this: ** **/type/fiscal_year/fiscal_period/entity ** **entity is the only one that is "optional" ** **When an entity is specified I get a return in a second or two.** Granted a ** **much smaller record set. ** **When entity is omitted I get about 50x the result count and that is where ** **I get to a 20 second return time. ** **Those results at first made me think the problem was io bound on the ** **router till I saw that the first return byte was so behind. ** **Am I misunderstanding your suggestion that additional indexes would ** **improve the speed at which the DB returns the data, because I am ** **interpreting your suggestion as one to improve the selection speed. ** **On Tue, Jun 6, 2017 at 4:18 PM, George Gergues ** **<[1][3]george.gergues at gmail.com> wrote: ** ** **** **For SQL table **add at least one index. it will improve table ** ** **scans. ** ** **** **On Jun 6, 2017 11:44, "Ken MacKenzie" <[1][2][4]ken at mack-z.com> wrote: ** ** **** ** **** **So I am in the demo and test phase of an early ReSTful API ** ** **for ** ** **** ** **reporting. ** ** **** ** **** **Currently a wider scale report set request hits the ** ** **following ** ** **** ** **marks: ** ** **** ** **** **TTFB: ~20s ** ** **** ** **** **Record Count: ~92k ** ** **** ** **** **Download Size: 15.8MB ** ** **** ** **** **Details: ** ** **** ** **** **Web Server: NGINX ** ** **** ** **** **Python App Server: Gunicorn ** ** **** ** **** **Web Framework: Falcon ** ** **** ** **** **Python version: 3.5 (in a venv) ** ** **** ** **** **DB: MS SQL Server Express using SQL Alchemy + pyodbc ** ** **** ** **** **Webserver OS: CentOS 7 ** ** **** ** **** **Gunicorn is setup with 4 workers, on a private port, nginx ** ** **does a ** ** **** ** **proxy ** ** **** ** **** **pass to the port ** ** **** ** **** **DB Details, the table in question has a total of about 8 ** ** **million ** ** **** ** **rows.** ** ** **** ** **** **Sample query execution within SQL Server Mgmt Studio is ~7s ** ** **** ** **** **So my question is which of the following would be a better ** ** **target ** ** **** ** **to ** ** **** ** **** **improve performance, or do I need to as my performance ** ** **should be ** ** **** ** **** **considered good enough.** I mean the server in this case is ** ** **a ** ** **** ** **surplus dual ** ** **** ** **** **core desktop right now. ** ** **** ** **** **add gzip compression to nginx for proxys ** ** **** ** **** **switch gunicorn to use a unix socket instead of a tcp port ** ** **** ** **** **consider leaner SQL and JSON marshaling requests instead of ** ** **ORM's ** ** **** ** **and ** ** **** ** **** **dictionary bundles. ** ** **** ** **** **Appreciate and advice or suggestions.** Thank you. ** ** **** ** **_______________________________________________ ** ** **** ** **TriZPUG mailing list ** ** **** ** **[2][3][5]TriZPUG at python.org ** ** **** ** **[3][4][6]https://mail.python.org/mailman/listinfo/trizpug ** ** **** ** **[4][5][7]http://tripython.org is the Triangle Python Users Group References ** **Visible links ** **1. mailto:[8]george.gergues at gmail.com ** **2. mailto:[9]ken at mack-z.com ** **3. mailto:[10]TriZPUG at python.org ** **4. [11]https://mail.python.org/mailman/listinfo/trizpug ** **5. [12]http://tripython.org/ _______________________________________________ TriZPUG mailing list [13]TriZPUG at python.org [14]https://mail.python.org/mailman/listinfo/trizpug [15]http://tripython.org is the Triangle Python Users Group References Visible links 1. https://pypi.python.org/pypi/profilehooks 2. mailto:ken at mack-z.com 3. mailto:george.gergues at gmail.com 4. mailto:ken at mack-z.com 5. mailto:TriZPUG at python.org 6. https://mail.python.org/mailman/listinfo/trizpug 7. http://tripython.org/ 8. mailto:george.gergues at gmail.com 9. mailto:ken at mack-z.com 10. mailto:TriZPUG at python.org 11. https://mail.python.org/mailman/listinfo/trizpug 12. http://tripython.org/ 13. mailto:TriZPUG at python.org 14. https://mail.python.org/mailman/listinfo/trizpug 15. http://tripython.org/ From ken at mack-z.com Tue Jun 6 16:47:26 2017 From: ken at mack-z.com (Ken MacKenzie) Date: Tue, 6 Jun 2017 16:47:26 -0400 Subject: [TriPython] Places to look at performance tuning In-Reply-To: References: Message-ID: I have played with some calls to cProfile, using in more of a call the server.py file I use for dev testing. And this comes to your question, yes using the ORM. Which I would wager could be the performance bottle neck as the record set returned grows. On Tue, Jun 6, 2017 at 4:44 PM, James Whisnant wrote: > I recommend profiling your code to see exactly where it is slow. It may > or > may not be what you expect. [1]https://pypi.python.org/ > pypi/profilehooks > is an easy to use decorate your functions to profile them. It does sound > like some of your database queries are slow. This may help you pinpoint > where and why. Are you using the SQLAlchemy ORM or Core? The ORM can > have > a large overhead (in some cases) as compared to the SQLAlchemy Core. > > On Tue, Jun 6, 2017 at 4:28 PM, Ken MacKenzie <[2]ken at mack-z.com> > wrote: > > ** **At present the SQL table has about 8 fields that make up the > primary key, > ** **that PK creates an index and is the only index at present. > ** **I don't think sql selection is my bottle neck. **here is why. > ** **basically my route is like this: > ** **/type/fiscal_year/fiscal_period/entity > ** **entity is the only one that is "optional" > ** **When an entity is specified I get a return in a second or two.** > Granted a > ** **much smaller record set. > ** **When entity is omitted I get about 50x the result count and that > is > where > ** **I get to a 20 second return time. > ** **Those results at first made me think the problem was io bound on > the > ** **router till I saw that the first return byte was so behind. > ** **Am I misunderstanding your suggestion that additional indexes > would > ** **improve the speed at which the DB returns the data, because I am > ** **interpreting your suggestion as one to improve the selection > speed. > ** **On Tue, Jun 6, 2017 at 4:18 PM, George Gergues > ** **<[1][3]george.gergues at gmail.com> wrote: > > ** ** **** **For SQL table **add at least one index. it will improve > table > ** ** **scans. > ** ** **** **On Jun 6, 2017 11:44, "Ken MacKenzie" > <[1][2][4]ken at mack-z.com> wrote: > > ** ** **** ** **** **So I am in the demo and test phase of an early > ReSTful API > ** ** **for > ** ** **** ** **reporting. > ** ** **** ** **** **Currently a wider scale report set request hits > the > ** ** **following > ** ** **** ** **marks: > ** ** **** ** **** **TTFB: ~20s > ** ** **** ** **** **Record Count: ~92k > ** ** **** ** **** **Download Size: 15.8MB > ** ** **** ** **** **Details: > ** ** **** ** **** **Web Server: NGINX > ** ** **** ** **** **Python App Server: Gunicorn > ** ** **** ** **** **Web Framework: Falcon > ** ** **** ** **** **Python version: 3.5 (in a venv) > ** ** **** ** **** **DB: MS SQL Server Express using SQL Alchemy + > pyodbc > ** ** **** ** **** **Webserver OS: CentOS 7 > ** ** **** ** **** **Gunicorn is setup with 4 workers, on a private > port, nginx > ** ** **does a > ** ** **** ** **proxy > ** ** **** ** **** **pass to the port > ** ** **** ** **** **DB Details, the table in question has a total of > about 8 > ** ** **million > ** ** **** ** **rows.** > ** ** **** ** **** **Sample query execution within SQL Server Mgmt > Studio is ~7s > ** ** **** ** **** **So my question is which of the following would > be a > better > ** ** **target > ** ** **** ** **to > ** ** **** ** **** **improve performance, or do I need to as my > performance > ** ** **should be > ** ** **** ** **** **considered good enough.** I mean the server in > this > case is > ** ** **a > ** ** **** ** **surplus dual > ** ** **** ** **** **core desktop right now. > ** ** **** ** **** **add gzip compression to nginx for proxys > ** ** **** ** **** **switch gunicorn to use a unix socket instead of a > tcp port > ** ** **** ** **** **consider leaner SQL and JSON marshaling requests > instead of > ** ** **ORM's > ** ** **** ** **and > ** ** **** ** **** **dictionary bundles. > ** ** **** ** **** **Appreciate and advice or suggestions.** Thank > you. > > ** ** **** ** **_______________________________________________ > ** ** **** ** **TriZPUG mailing list > ** ** **** ** **[2][3][5]TriZPUG at python.org > ** ** **** ** > **[3][4][6]https://mail.python.org/mailman/listinfo/trizpug > ** ** **** ** **[4][5][7]http://tripython.org is the Triangle Python > Users Group > > References > > ** **Visible links > ** **1. mailto:[8]george.gergues at gmail.com > ** **2. mailto:[9]ken at mack-z.com > ** **3. mailto:[10]TriZPUG at python.org > ** **4. [11]https://mail.python.org/mailman/listinfo/trizpug > ** **5. [12]http://tripython.org/ > > _______________________________________________ > TriZPUG mailing list > [13]TriZPUG at python.org > [14]https://mail.python.org/mailman/listinfo/trizpug > [15]http://tripython.org is the Triangle Python Users Group > > p > > -------------- next part -------------- I have played with some calls to cProfile, using in more of a call the server.py file I use for dev testing. And this comes to your question, yes using the ORM.** Which I would wager could be the performance bottle neck as the record set returned grows. On Tue, Jun 6, 2017 at 4:44 PM, James Whisnant <[1]jwhisnant at gmail.com> wrote: ** **I recommend profiling your code to see exactly where it is slow. It may or ** **may not be what you expect. [1][2]https://pypi.python.org/pypi/profilehooks ** **is an easy to use decorate your functions to profile them. It does sound ** **like some of your database queries are slow. This may help you pinpoint ** **where and why. Are you using the SQLAlchemy ORM or Core? The ORM can have ** **a large overhead (in some cases) as compared to the SQLAlchemy Core. ** **On Tue, Jun 6, 2017 at 4:28 PM, Ken MacKenzie <[2][3]ken at mack-z.com> wrote: ** ** **** **At present the SQL table has about 8 fields that make up the ** ** **primary key, ** ** **** **that PK creates an index and is the only index at present. ** ** **** **I don't think sql selection is my bottle neck. **here is why. ** ** **** **basically my route is like this: ** ** **** **/type/fiscal_year/fiscal_period/entity ** ** **** **entity is the only one that is "optional" ** ** **** **When an entity is specified I get a return in a second or two.** ** ** **Granted a ** ** **** **much smaller record set. ** ** **** **When entity is omitted I get about 50x the result count and that is ** ** **where ** ** **** **I get to a 20 second return time. ** ** **** **Those results at first made me think the problem was io bound on ** ** **the ** ** **** **router till I saw that the first return byte was so behind. ** ** **** **Am I misunderstanding your suggestion that additional indexes would ** ** **** **improve the speed at which the DB returns the data, because I am ** ** **** **interpreting your suggestion as one to improve the selection speed. ** ** **** **On Tue, Jun 6, 2017 at 4:18 PM, George Gergues ** ** **** **<[1][3][4]george.gergues at gmail.com> wrote: ** ** **** ** **** **For SQL table **add at least one index. it will improve ** ** **table ** ** **** ** **scans. ** ** **** ** **** **On Jun 6, 2017 11:44, "Ken MacKenzie" ** ** **<[1][2][4][5]ken at mack-z.com> wrote: ** ** **** ** **** ** **** **So I am in the demo and test phase of an early ** ** **ReSTful API ** ** **** ** **for ** ** **** ** **** ** **reporting. ** ** **** ** **** ** **** **Currently a wider scale report set request hits the ** ** **** ** **following ** ** **** ** **** ** **marks: ** ** **** ** **** ** **** **TTFB: ~20s ** ** **** ** **** ** **** **Record Count: ~92k ** ** **** ** **** ** **** **Download Size: 15.8MB ** ** **** ** **** ** **** **Details: ** ** **** ** **** ** **** **Web Server: NGINX ** ** **** ** **** ** **** **Python App Server: Gunicorn ** ** **** ** **** ** **** **Web Framework: Falcon ** ** **** ** **** ** **** **Python version: 3.5 (in a venv) ** ** **** ** **** ** **** **DB: MS SQL Server Express using SQL Alchemy + ** ** **pyodbc ** ** **** ** **** ** **** **Webserver OS: CentOS 7 ** ** **** ** **** ** **** **Gunicorn is setup with 4 workers, on a private ** ** **port, nginx ** ** **** ** **does a ** ** **** ** **** ** **proxy ** ** **** ** **** ** **** **pass to the port ** ** **** ** **** ** **** **DB Details, the table in question has a total of ** ** **about 8 ** ** **** ** **million ** ** **** ** **** ** **rows.** ** ** **** ** **** ** **** **Sample query execution within SQL Server Mgmt ** ** **Studio is ~7s ** ** **** ** **** ** **** **So my question is which of the following would be a ** ** **better ** ** **** ** **target ** ** **** ** **** ** **to ** ** **** ** **** ** **** **improve performance, or do I need to as my ** ** **performance ** ** **** ** **should be ** ** **** ** **** ** **** **considered good enough.** I mean the server in this ** ** **case is ** ** **** ** **a ** ** **** ** **** ** **surplus dual ** ** **** ** **** ** **** **core desktop right now. ** ** **** ** **** ** **** **add gzip compression to nginx for proxys ** ** **** ** **** ** **** **switch gunicorn to use a unix socket instead of a ** ** **tcp port ** ** **** ** **** ** **** **consider leaner SQL and JSON marshaling requests ** ** **instead of ** ** **** ** **ORM's ** ** **** ** **** ** **and ** ** **** ** **** ** **** **dictionary bundles. ** ** **** ** **** ** **** **Appreciate and advice or suggestions.** Thank you. ** ** **** ** **** ** **_______________________________________________ ** ** **** ** **** ** **TriZPUG mailing list ** ** **** ** **** ** **[2][3][5][6]TriZPUG at python.org ** ** **** ** **** ** ** ** ****[3][4][6][7]https://mail.python.org/mailman/listinfo/trizpug ** ** **** ** **** ** **[4][5][7][8]http://tripython.org is the Triangle Python ** ** **Users Group ** ** **References ** ** **** **Visible links ** ** **** **1. mailto:[8][9]george.gergues at gmail.com ** ** **** **2. mailto:[9][10]ken at mack-z.com ** ** **** **3. mailto:[10][11]TriZPUG at python.org ** ** **** **4. [11][12]https://mail.python.org/mailman/listinfo/trizpug ** ** **** **5. [12][13]http://tripython.org/ ** ** **_______________________________________________ ** ** **TriZPUG mailing list ** ** **[13][14]TriZPUG at python.org ** ** **[14][15]https://mail.python.org/mailman/listinfo/trizpug ** ** **[15][16]http://tripython.org is the Triangle Python Users Group p References Visible links 1. mailto:jwhisnant at gmail.com 2. https://pypi.python.org/pypi/profilehooks 3. mailto:ken at mack-z.com 4. mailto:george.gergues at gmail.com 5. mailto:ken at mack-z.com 6. mailto:TriZPUG at python.org 7. https://mail.python.org/mailman/listinfo/trizpug 8. http://tripython.org/ 9. mailto:george.gergues at gmail.com 10. mailto:ken at mack-z.com 11. mailto:TriZPUG at python.org 12. https://mail.python.org/mailman/listinfo/trizpug 13. http://tripython.org/ 14. mailto:TriZPUG at python.org 15. https://mail.python.org/mailman/listinfo/trizpug 16. http://tripython.org/ From ken at mack-z.com Tue Jun 6 17:22:58 2017 From: ken at mack-z.com (Ken MacKenzie) Date: Tue, 6 Jun 2017 17:22:58 -0400 Subject: [TriPython] Places to look at performance tuning In-Reply-To: References: Message-ID: (method 'poll' of 'select.poll objects) seems to be the place where I am hitting the performance bottleneck. To me I could think of that as either the ORM or the previous indexing suggestion or both. On Tue, Jun 6, 2017 at 4:47 PM, Ken MacKenzie wrote: > I have played with some calls to cProfile, using in more of a call the > server.py file I use for dev testing. > > And this comes to your question, yes using the ORM. Which I would wager > could be the performance bottle neck as the record set returned grows. > > On Tue, Jun 6, 2017 at 4:44 PM, James Whisnant > wrote: > >> I recommend profiling your code to see exactly where it is slow. It >> may or >> may not be what you expect. [1]https://pypi.python.org/pyp >> i/profilehooks >> is an easy to use decorate your functions to profile them. It does >> sound >> like some of your database queries are slow. This may help you pinpoint >> where and why. Are you using the SQLAlchemy ORM or Core? The ORM can >> have >> a large overhead (in some cases) as compared to the SQLAlchemy Core. >> >> On Tue, Jun 6, 2017 at 4:28 PM, Ken MacKenzie <[2]ken at mack-z.com> >> wrote: >> >> ** **At present the SQL table has about 8 fields that make up the >> primary key, >> ** **that PK creates an index and is the only index at present. >> ** **I don't think sql selection is my bottle neck. **here is why. >> ** **basically my route is like this: >> ** **/type/fiscal_year/fiscal_period/entity >> ** **entity is the only one that is "optional" >> ** **When an entity is specified I get a return in a second or two.** >> Granted a >> ** **much smaller record set. >> ** **When entity is omitted I get about 50x the result count and >> that is >> where >> ** **I get to a 20 second return time. >> ** **Those results at first made me think the problem was io bound on >> the >> ** **router till I saw that the first return byte was so behind. >> ** **Am I misunderstanding your suggestion that additional indexes >> would >> ** **improve the speed at which the DB returns the data, because I am >> ** **interpreting your suggestion as one to improve the selection >> speed. >> ** **On Tue, Jun 6, 2017 at 4:18 PM, George Gergues >> ** **<[1][3]george.gergues at gmail.com> wrote: >> >> ** ** **** **For SQL table **add at least one index. it will improve >> table >> ** ** **scans. >> ** ** **** **On Jun 6, 2017 11:44, "Ken MacKenzie" >> <[1][2][4]ken at mack-z.com> wrote: >> >> ** ** **** ** **** **So I am in the demo and test phase of an early >> ReSTful API >> ** ** **for >> ** ** **** ** **reporting. >> ** ** **** ** **** **Currently a wider scale report set request hits >> the >> ** ** **following >> ** ** **** ** **marks: >> ** ** **** ** **** **TTFB: ~20s >> ** ** **** ** **** **Record Count: ~92k >> ** ** **** ** **** **Download Size: 15.8MB >> ** ** **** ** **** **Details: >> ** ** **** ** **** **Web Server: NGINX >> ** ** **** ** **** **Python App Server: Gunicorn >> ** ** **** ** **** **Web Framework: Falcon >> ** ** **** ** **** **Python version: 3.5 (in a venv) >> ** ** **** ** **** **DB: MS SQL Server Express using SQL Alchemy + >> pyodbc >> ** ** **** ** **** **Webserver OS: CentOS 7 >> ** ** **** ** **** **Gunicorn is setup with 4 workers, on a private >> port, nginx >> ** ** **does a >> ** ** **** ** **proxy >> ** ** **** ** **** **pass to the port >> ** ** **** ** **** **DB Details, the table in question has a total of >> about 8 >> ** ** **million >> ** ** **** ** **rows.** >> ** ** **** ** **** **Sample query execution within SQL Server Mgmt >> Studio is ~7s >> ** ** **** ** **** **So my question is which of the following would >> be a >> better >> ** ** **target >> ** ** **** ** **to >> ** ** **** ** **** **improve performance, or do I need to as my >> performance >> ** ** **should be >> ** ** **** ** **** **considered good enough.** I mean the server in >> this >> case is >> ** ** **a >> ** ** **** ** **surplus dual >> ** ** **** ** **** **core desktop right now. >> ** ** **** ** **** **add gzip compression to nginx for proxys >> ** ** **** ** **** **switch gunicorn to use a unix socket instead of >> a >> tcp port >> ** ** **** ** **** **consider leaner SQL and JSON marshaling requests >> instead of >> ** ** **ORM's >> ** ** **** ** **and >> ** ** **** ** **** **dictionary bundles. >> ** ** **** ** **** **Appreciate and advice or suggestions.** Thank >> you. >> >> ** ** **** ** **_______________________________________________ >> ** ** **** ** **TriZPUG mailing list >> ** ** **** ** **[2][3][5]TriZPUG at python.org >> ** ** **** ** >> **[3][4][6]https://mail.python.org/mailman/listinfo/trizpug >> ** ** **** ** **[4][5][7]http://tripython.org is the Triangle Python >> Users Group >> >> References >> > > -------------- next part -------------- (method 'poll' of 'select.poll objects) seems to be the place where I am hitting the performance bottleneck. To me I could think of that as either the ORM or the previous indexing suggestion or both. On Tue, Jun 6, 2017 at 4:47 PM, Ken MacKenzie <[1]ken at mack-z.com> wrote: I have played with some calls to cProfile, using in more of a call the server.py file I use for dev testing. And this comes to your question, yes using the ORM.** Which I would wager could be the performance bottle neck as the record set returned grows. On Tue, Jun 6, 2017 at 4:44 PM, James Whisnant <[2]jwhisnant at gmail.com> wrote: ** **I recommend profiling your code to see exactly where it is slow. It may or ** **may not be what you expect. [1][3]https://pypi.python.org/pypi/profilehooks ** **is an easy to use decorate your functions to profile them. It does sound ** **like some of your database queries are slow. This may help you pinpoint ** **where and why. Are you using the SQLAlchemy ORM or Core? The ORM can have ** **a large overhead (in some cases) as compared to the SQLAlchemy Core. ** **On Tue, Jun 6, 2017 at 4:28 PM, Ken MacKenzie <[2][4]ken at mack-z.com> wrote: ** ** **** **At present the SQL table has about 8 fields that make up the ** ** **primary key, ** ** **** **that PK creates an index and is the only index at present. ** ** **** **I don't think sql selection is my bottle neck. **here is why. ** ** **** **basically my route is like this: ** ** **** **/type/fiscal_year/fiscal_period/entity ** ** **** **entity is the only one that is "optional" ** ** **** **When an entity is specified I get a return in a second or two.** ** ** **Granted a ** ** **** **much smaller record set. ** ** **** **When entity is omitted I get about 50x the result count and that is ** ** **where ** ** **** **I get to a 20 second return time. ** ** **** **Those results at first made me think the problem was io bound on ** ** **the ** ** **** **router till I saw that the first return byte was so behind. ** ** **** **Am I misunderstanding your suggestion that additional indexes would ** ** **** **improve the speed at which the DB returns the data, because I am ** ** **** **interpreting your suggestion as one to improve the selection speed. ** ** **** **On Tue, Jun 6, 2017 at 4:18 PM, George Gergues ** ** **** **<[1][3][5]george.gergues at gmail.com> wrote: ** ** **** ** **** **For SQL table **add at least one index. it will improve ** ** **table ** ** **** ** **scans. ** ** **** ** **** **On Jun 6, 2017 11:44, "Ken MacKenzie" ** ** **<[1][2][4][6]ken at mack-z.com> wrote: ** ** **** ** **** ** **** **So I am in the demo and test phase of an early ** ** **ReSTful API ** ** **** ** **for ** ** **** ** **** ** **reporting. ** ** **** ** **** ** **** **Currently a wider scale report set request hits the ** ** **** ** **following ** ** **** ** **** ** **marks: ** ** **** ** **** ** **** **TTFB: ~20s ** ** **** ** **** ** **** **Record Count: ~92k ** ** **** ** **** ** **** **Download Size: 15.8MB ** ** **** ** **** ** **** **Details: ** ** **** ** **** ** **** **Web Server: NGINX ** ** **** ** **** ** **** **Python App Server: Gunicorn ** ** **** ** **** ** **** **Web Framework: Falcon ** ** **** ** **** ** **** **Python version: 3.5 (in a venv) ** ** **** ** **** ** **** **DB: MS SQL Server Express using SQL Alchemy + ** ** **pyodbc ** ** **** ** **** ** **** **Webserver OS: CentOS 7 ** ** **** ** **** ** **** **Gunicorn is setup with 4 workers, on a private ** ** **port, nginx ** ** **** ** **does a ** ** **** ** **** ** **proxy ** ** **** ** **** ** **** **pass to the port ** ** **** ** **** ** **** **DB Details, the table in question has a total of ** ** **about 8 ** ** **** ** **million ** ** **** ** **** ** **rows.** ** ** **** ** **** ** **** **Sample query execution within SQL Server Mgmt ** ** **Studio is ~7s ** ** **** ** **** ** **** **So my question is which of the following would be a ** ** **better ** ** **** ** **target ** ** **** ** **** ** **to ** ** **** ** **** ** **** **improve performance, or do I need to as my ** ** **performance ** ** **** ** **should be ** ** **** ** **** ** **** **considered good enough.** I mean the server in this ** ** **case is ** ** **** ** **a ** ** **** ** **** ** **surplus dual ** ** **** ** **** ** **** **core desktop right now. ** ** **** ** **** ** **** **add gzip compression to nginx for proxys ** ** **** ** **** ** **** **switch gunicorn to use a unix socket instead of a ** ** **tcp port ** ** **** ** **** ** **** **consider leaner SQL and JSON marshaling requests ** ** **instead of ** ** **** ** **ORM's ** ** **** ** **** ** **and ** ** **** ** **** ** **** **dictionary bundles. ** ** **** ** **** ** **** **Appreciate and advice or suggestions.** Thank you. ** ** **** ** **** ** **_______________________________________________ ** ** **** ** **** ** **TriZPUG mailing list ** ** **** ** **** ** **[2][3][5][7]TriZPUG at python.org ** ** **** ** **** ** ** ** ****[3][4][6][8]https://mail.python.org/mailman/listinfo/trizpug ** ** **** ** **** ** **[4][5][7][9]http://tripython.org is the Triangle Python ** ** **Users Group ** ** **References References Visible links 1. mailto:ken at mack-z.com 2. mailto:jwhisnant at gmail.com 3. https://pypi.python.org/pypi/profilehooks 4. mailto:ken at mack-z.com 5. mailto:george.gergues at gmail.com 6. mailto:ken at mack-z.com 7. mailto:TriZPUG at python.org 8. https://mail.python.org/mailman/listinfo/trizpug 9. http://tripython.org/ From ken at mack-z.com Tue Jun 6 17:31:57 2017 From: ken at mack-z.com (Ken MacKenzie) Date: Tue, 6 Jun 2017 17:31:57 -0400 Subject: [TriPython] Places to look at performance tuning In-Reply-To: References: Message-ID: Wait brain fart, that actually goes back to the IO bound issue. Ok so let me think about this. If it is IO bound why does it still show near 20 seconds for TTFB? steps, rough outline take request based on route send request to function 1 function 1 adds some gravy then redirects to a core function core function processes the query string in the request and... does the ORM filter and groub_by setup handles setting up the DictBundle for now also deals with date/time and decimal conversion on fields gets the data after all the data is back marshals it for the return (json by default but there is also a csv option) returns to function 1 function 1 places returned marshaled data in resp.body and creates the resp.header When I was thinking IO bound I was thinking that it was the browser actually getting the response data over http. Perhaps my IO bound issues are in the creation of the objects or the marshaling the response to explain function 1 and core. core is a core transaction module, functions 1-3 are types of transactions expense, revenue, budget hence that division in the logic. On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie wrote: > > (method 'poll' of 'select.poll objects) seems to be the place where I am > hitting the performance bottleneck. > > To me I could think of that as either the ORM or the previous indexing > suggestion or both. > > > -------------- next part -------------- Wait brain fart, that actually goes back to the IO bound issue. Ok so let me think about this.** If it is IO bound why does it still show near 20 seconds for TTFB? steps, rough outline take request based on route send request to function 1 function 1 adds some gravy then redirects to a core function core function processes the query string in the request and... ** ** does the ORM filter and groub_by setup ** ** handles setting up the DictBundle ** ** for now also deals with date/time and decimal conversion on fields ** ** gets the data ** ** after all the data is back marshals it for the return (json by default but there is also a csv option) ** ** returns to function 1 function 1 places returned marshaled data in resp.body and creates the resp.header When I was thinking IO bound I was thinking that it was the browser actually getting the response data over http. Perhaps my IO bound issues are in the creation of the objects or the marshaling the response to explain function **1 and core. **core is a core transaction module, functions 1-3 are types of transactions expense, revenue, budget hence that division in the logic. On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie <[1]ken at mack-z.com> wrote: (method 'poll' of 'select.poll objects) seems to be the place where I am hitting the performance bottleneck. To me I could think of that as either the ORM or the previous indexing suggestion or both. References Visible links 1. mailto:ken at mack-z.com From david at handysoftware.com Tue Jun 6 22:28:15 2017 From: david at handysoftware.com (David Handy) Date: Tue, 6 Jun 2017 22:28:15 -0400 (EDT) Subject: [TriPython] Places to look at performance tuning In-Reply-To: References: Message-ID: <1496802495.062923157@apps.rackspace.com> Hi Ken - Your previous email said your web server is running on CentOS and your database is MS SQL Server Express (running on a Windows server, presumably.) What does the CPU usage look like on your web server and database server, respectively, during a test? (It would probably help to have a test where you do several consecutive requests, to make CPU usage more obvious.) David H On Tuesday, June 6, 2017 5:31pm, "Ken MacKenzie" said: > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > https://mail.python.org/mailman/listinfo/trizpug > http://tripython.org is the Triangle Python Users Group > Wait brain fart, that actually goes back to the IO bound issue. > > Ok so let me think about this. If it is IO bound why does it still show > near 20 seconds for TTFB? > > steps, rough outline > > take request > based on route send request to function 1 > function 1 adds some gravy then redirects to a core function > core function processes the query string in the request and... > does the ORM filter and groub_by setup > handles setting up the DictBundle > for now also deals with date/time and decimal conversion on fields > gets the data > after all the data is back marshals it for the return (json by default > but there is also a csv option) > returns to function 1 > function 1 places returned marshaled data in resp.body and creates the > resp.header > > When I was thinking IO bound I was thinking that it was the browser > actually getting the response data over http. > > Perhaps my IO bound issues are in the creation of the objects or the > marshaling the response > > to explain function 1 and core. core is a core transaction module, > functions 1-3 are types of transactions expense, revenue, budget hence that > division in the logic. > > > On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie wrote: > > > > > (method 'poll' of 'select.poll objects) seems to be the place where I am > > hitting the performance bottleneck. > > > > To me I could think of that as either the ORM or the previous indexing > > suggestion or both. > > > > > > > Wait brain fart, that actually goes back to the IO bound issue. > Ok so let me think about this.** If it is IO bound why does it still show > near 20 seconds for TTFB? > steps, rough outline > take request > based on route send request to function 1 > function 1 adds some gravy then redirects to a core function > core function processes the query string in the request and... > ** ** does the ORM filter and groub_by setup > ** ** handles setting up the DictBundle > ** ** for now also deals with date/time and decimal conversion on fields > ** ** gets the data > ** ** after all the data is back marshals it for the return (json by > default but there is also a csv option) > ** ** returns to function 1 > function 1 places returned marshaled data in resp.body and creates the > resp.header > When I was thinking IO bound I was thinking that it was the browser > actually getting the response data over http. > Perhaps my IO bound issues are in the creation of the objects or the > marshaling the response > to explain function **1 and core. **core is a core transaction module, > functions 1-3 are types of transactions expense, revenue, budget hence > that division in the logic. > On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie <[1]ken at mack-z.com> wrote: > > (method 'poll' of 'select.poll objects) seems to be the place where I am > hitting the performance bottleneck. > To me I could think of that as either the ORM or the previous indexing > suggestion or both. > > References > > Visible links > 1. mailto:ken at mack-z.com > -------------- next part -------------- Hi Ken - Your previous email said your web server is running on CentOS and your database is MS SQL Server Express (running on a Windows server, presumably.) What does the CPU usage look like on your web server and database server, respectively, during a test? (It would probably help to have a test where you do several consecutive requests, to make CPU usage more obvious.) David H On Tuesday, June 6, 2017 5:31pm, "Ken MacKenzie" said: > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > https://mail.python.org/mailman/listinfo/trizpug > http://tripython.org is the Triangle Python Users Group > Wait brain fart, that actually goes back to the IO bound issue. > > Ok so let me think about this. If it is IO bound why does it still show > near 20 seconds for TTFB? > > steps, rough outline > > take request > based on route send request to function 1 > function 1 adds some gravy then redirects to a core function > core function processes the query string in the request and... > does the ORM filter and groub_by setup > handles setting up the DictBundle > for now also deals with date/time and decimal conversion on fields > gets the data > after all the data is back marshals it for the return (json by default > but there is also a csv option) > returns to function 1 > function 1 places returned marshaled data in resp.body and creates the > resp.header > > When I was thinking IO bound I was thinking that it was the browser > actually getting the response data over http. > > Perhaps my IO bound issues are in the creation of the objects or the > marshaling the response > > to explain function 1 and core. core is a core transaction module, > functions 1-3 are types of transactions expense, revenue, budget hence that > division in the logic. > > > On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie wrote: > > > > > (method 'poll' of 'select.poll objects) seems to be the place where I am > > hitting the performance bottleneck. > > > > To me I could think of that as either the ORM or the previous indexing > > suggestion or both. > > > > > > > Wait brain fart, that actually goes back to the IO bound issue. > Ok so let me think about this.** If it is IO bound why does it still show > near 20 seconds for TTFB? > steps, rough outline > take request > based on route send request to function 1 > function 1 adds some gravy then redirects to a core function > core function processes the query string in the request and... > ** ** does the ORM filter and groub_by setup > ** ** handles setting up the DictBundle > ** ** for now also deals with date/time and decimal conversion on fields > ** ** gets the data > ** ** after all the data is back marshals it for the return (json by > default but there is also a csv option) > ** ** returns to function 1 > function 1 places returned marshaled data in resp.body and creates the > resp.header > When I was thinking IO bound I was thinking that it was the browser > actually getting the response data over http. > Perhaps my IO bound issues are in the creation of the objects or the > marshaling the response > to explain function **1 and core. **core is a core transaction module, > functions 1-3 are types of transactions expense, revenue, budget hence > that division in the logic. > On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie <[1]ken at mack-z.com> wrote: > > (method 'poll' of 'select.poll objects) seems to be the place where I am > hitting the performance bottleneck. > To me I could think of that as either the ORM or the previous indexing > suggestion or both. > > References > > Visible links > 1. mailto:ken at mack-z.com > From david at handysoftware.com Tue Jun 6 22:31:10 2017 From: david at handysoftware.com (David Handy) Date: Tue, 6 Jun 2017 22:31:10 -0400 (EDT) Subject: [TriPython] Places to look at performance tuning In-Reply-To: <1496802495.062923157@apps.rackspace.com> References: <1496802495.062923157@apps.rackspace.com> Message-ID: <1496802670.746326531@apps.rackspace.com> By the way, your I/O waiting could be on database sockets as well as on communication with the browser. On Tuesday, June 6, 2017 10:28pm, "David Handy" said: Hi Ken - Your previous email said your web server is running on CentOS and your database is MS SQL Server Express (running on a Windows server, presumably.) What does the CPU usage look like on your web server and database server, respectively, during a test? (It would probably help to have a test where you do several consecutive requests, to make CPU usage more obvious.) David H On Tuesday, June 6, 2017 5:31pm, "Ken MacKenzie" said: > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > https://mail.python.org/mailman/listinfo/trizpug > http://tripython.org is the Triangle Python Users Group > Wait brain fart, that actually goes back to the IO bound issue. > > Ok so let me think about this. If it is IO bound why does it still show > near 20 seconds for TTFB? > > steps, rough outline > > take request > based on route send request to function 1 > function 1 adds some gravy then redirects to a core function > core function processes the query string in the request and... > does the ORM filter and groub_by setup > handles setting up the DictBundle > for now also deals with date/time and decimal conversion on fields > gets the data > after all the data is back marshals it for the return (json by default > but there is also a csv option) > returns to function 1 > function 1 places returned marshaled data in resp.body and creates the > resp.header > > When I was thinking IO bound I was thinking that it was the browser > actually getting the response data over http. > > Perhaps my IO bound issues are in the creation of the objects or the > marshaling the response > > to explain function 1 and core. core is a core transaction module, > functions 1-3 are types of transactions expense, revenue, budget hence that > division in the logic. > > > On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie wrote: > > > > > (method 'poll' of 'select.poll objects) seems to be the place where I am > > hitting the performance bottleneck. > > > > To me I could think of that as either the ORM or the previous indexing > > suggestion or both. > > > > > > > Wait brain fart, that actually goes back to the IO bound issue. > Ok so let me think about this.** If it is IO bound why does it still show > near 20 seconds for TTFB? > steps, rough outline > take request > based on route send request to function 1 > function 1 adds some gravy then redirects to a core function > core function processes the query string in the request and... > ** ** does the ORM filter and groub_by setup > ** ** handles setting up the DictBundle > ** ** for now also deals with date/time and decimal conversion on fields > ** ** gets the data > ** ** after all the data is back marshals it for the return (json by > default but there is also a csv option) > ** ** returns to function 1 > function 1 places returned marshaled data in resp.body and creates the > resp.header > When I was thinking IO bound I was thinking that it was the browser > actually getting the response data over http. > Perhaps my IO bound issues are in the creation of the objects or the > marshaling the response > to explain function **1 and core. **core is a core transaction module, > functions 1-3 are types of transactions expense, revenue, budget hence > that division in the logic. > On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie <[1]ken at mack-z.com> wrote: > > (method 'poll' of 'select.poll objects) seems to be the place where I am > hitting the performance bottleneck. > To me I could think of that as either the ORM or the previous indexing > suggestion or both. > > References > > Visible links > 1. mailto:ken at mack-z.com > -------------- next part -------------- By the way, your I/O waiting could be on database sockets as well as on communication with the browser. On Tuesday, June 6, 2017 10:28pm, "David Handy" said: Hi Ken - Your previous email said your web server is running on CentOS and your database is MS SQL Server Express (running on a Windows server, presumably.) What does the CPU usage look like on your web server and database server, respectively, during a test? (It would probably help to have a test where you do several consecutive requests, to make CPU usage more obvious.) David H On Tuesday, June 6, 2017 5:31pm, "Ken MacKenzie" said: > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > https://mail.python.org/mailman/listinfo/trizpug > http://tripython.org is the Triangle Python Users Group > Wait brain fart, that actually goes back to the IO bound issue. > > Ok so let me think about this. If it is IO bound why does it still show > near 20 seconds for TTFB? > > steps, rough outline > > take request > based on route send request to function 1 > function 1 adds some gravy then redirects to a core function > core function processes the query string in the request and... > does the ORM filter and groub_by setup > handles setting up the DictBundle > for now also deals with date/time and decimal conversion on fields > gets the data > after all the data is back marshals it for the return (json by default > but there is also a csv option) > returns to function 1 > function 1 places returned marshaled data in resp.body and creates the > resp.header > > When I was thinking IO bound I was thinking that it was the browser > actually getting the response data over http. > > Perhaps my IO bound issues are in the creation of the objects or the > marshaling the response > > to explain function 1 and core. core is a core transaction module, > functions 1-3 are types of transactions expense, revenue, budget hence that > division in the logic. > > > On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie wrote: > > > > > (method 'poll' of 'select.poll objects) seems to be the place where I am > > hitting the performance bottleneck. > > > > To me I could think of that as either the ORM or the previous indexing > > suggestion or both. > > > > > > > Wait brain fart, that actually goes back to the IO bound issue. > Ok so let me think about this.** If it is IO bound why does it still show > near 20 seconds for TTFB? > steps, rough outline > take request > based on route send request to function 1 > function 1 adds some gravy then redirects to a core function > core function processes the query string in the request and... > ** ** does the ORM filter and groub_by setup > ** ** handles setting up the DictBundle > ** ** for now also deals with date/time and decimal conversion on fields > ** ** gets the data > ** ** after all the data is back marshals it for the return (json by > default but there is also a csv option) > ** ** returns to function 1 > function 1 places returned marshaled data in resp.body and creates the > resp.header > When I was thinking IO bound I was thinking that it was the browser > actually getting the response data over http. > Perhaps my IO bound issues are in the creation of the objects or the > marshaling the response > to explain function **1 and core. **core is a core transaction module, > functions 1-3 are types of transactions expense, revenue, budget hence > that division in the logic. > On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie <[1]ken at mack-z.com> wrote: > > (method 'poll' of 'select.poll objects) seems to be the place where I am > hitting the performance bottleneck. > To me I could think of that as either the ORM or the previous indexing > suggestion or both. > > References > > Visible links > 1. mailto:ken at mack-z.com > From cbc at unc.edu Wed Jun 7 10:24:29 2017 From: cbc at unc.edu (Calloway, Chris) Date: Wed, 7 Jun 2017 14:24:29 +0000 Subject: [TriPython] RENCI is Hiring Message-ID: <260203EC-74B8-4163-BD8E-4883F7F57EB1@unc.edu> https://unc.peopleadmin.com/postings/121621 Come be my co-worker. I?m trying to get HR to post the salary range. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 -------------- next part -------------- [1]https://unc.peopleadmin.com/postings/121621 Come be my co-worker. I'm trying to get HR to post the salary range. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 References Visible links 1. https://unc.peopleadmin.com/postings/121621 From cbc at unc.edu Wed Jun 7 11:04:18 2017 From: cbc at unc.edu (Calloway, Chris) Date: Wed, 7 Jun 2017 15:04:18 +0000 Subject: [TriPython] RENCI is Hiring Message-ID: <7EDA1F3E-2176-48DE-8D87-22E307E140F0@unc.edu> $65,205 - $136,920, ?dependent on experience.? This is an ?EHRA Non-Faculty? position, which means vacation time of 5 weeks per year starting the very first year. Great health insurance. Great retirement options. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 On 6/7/17, 10:24 AM, "TriZPUG on behalf of Calloway, Chris" wrote: https://unc.peopleadmin.com/postings/121621 Come be my co-worker. I?m trying to get HR to post the salary range. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 From ken at mack-z.com Wed Jun 7 17:09:51 2017 From: ken at mack-z.com (Ken MacKenzie) Date: Wed, 7 Jun 2017 17:09:51 -0400 Subject: [TriPython] Places to look at performance tuning In-Reply-To: <1496802670.746326531@apps.rackspace.com> References: <1496802495.062923157@apps.rackspace.com> <1496802670.746326531@apps.rackspace.com> Message-ID: OK so here is what we got so far. Code: I moved from the ORM model to using the expression language. Also skipped the DictBundle but my marshaling statement is still along the lines of return json.dumps([dict(r) for r in records] default alchemyencode) Something like that. No discernible performance improvement, odd. cProfile shows the biggest time spent in the fetchone process iterating through I am guessing in the above statement. So watching CentOS box we barely tick above 12% cpu for this Then we get to Windows. So I click a refresh on the resource. Immediately CPU goes up to 40%, network traffic matches, and they stay at that level for the main duration of the run. But what about memory. Well the VM in question has a ll of 2gb of ram and frankly it is at 95% baseline. I think we are getting to the problem here. So the sql server side is working for all but the last 2 seconds of the request return, and the whole time memory is basically maxed. So that might be the issue... On Tue, Jun 6, 2017 at 10:31 PM, David Handy wrote: > By the way, your I/O waiting could be on database sockets as well as on > communication with the browser. > > > > On Tuesday, June 6, 2017 10:28pm, "David Handy" < > david at handysoftware.com> > said: > > Hi Ken - > > > > Your previous email said your web server is running on CentOS and your > database is MS SQL Server Express (running on a Windows server, > presumably.) > > > > What does the CPU usage look like on your web server and database > server, > respectively, during a test? (It would probably help to have a test > where > you do several consecutive requests, to make CPU usage more obvious.) > > > > David H > > > > On Tuesday, June 6, 2017 5:31pm, "Ken MacKenzie" said: > > > _______________________________________________ > > TriZPUG mailing list > > TriZPUG at python.org > > https://mail.python.org/mailman/listinfo/trizpug > > http://tripython.org is the Triangle Python Users Group > > Wait brain fart, that actually goes back to the IO bound issue. > > > > Ok so let me think about this. If it is IO bound why does it still > show > > near 20 seconds for TTFB? > > > > steps, rough outline > > > > take request > > based on route send request to function 1 > > function 1 adds some gravy then redirects to a core function > > core function processes the query string in the request and... > > does the ORM filter and groub_by setup > > handles setting up the DictBundle > > for now also deals with date/time and decimal conversion on fields > > gets the data > > after all the data is back marshals it for the return (json by default > > but there is also a csv option) > > returns to function 1 > > function 1 places returned marshaled data in resp.body and creates the > > resp.header > > > > When I was thinking IO bound I was thinking that it was the browser > > actually getting the response data over http. > > > > Perhaps my IO bound issues are in the creation of the objects or the > > marshaling the response > > > > to explain function 1 and core. core is a core transaction module, > > functions 1-3 are types of transactions expense, revenue, budget hence > that > > division in the logic. > > > > > > On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie wrote: > > > > > > > > (method 'poll' of 'select.poll objects) seems to be the place where > I > am > > > hitting the performance bottleneck. > > > > > > To me I could think of that as either the ORM or the previous > indexing > > > suggestion or both. > > > > > > > > > > > Wait brain fart, that actually goes back to the IO bound issue. > > Ok so let me think about this.** If it is IO bound why does it still > show > > near 20 seconds for TTFB? > > steps, rough outline > > take request > > based on route send request to function 1 > > function 1 adds some gravy then redirects to a core function > > core function processes the query string in the request and... > > ** ** does the ORM filter and groub_by setup > > ** ** handles setting up the DictBundle > > ** ** for now also deals with date/time and decimal conversion on > fields > > ** ** gets the data > > ** ** after all the data is back marshals it for the return (json by > > default but there is also a csv option) > > ** ** returns to function 1 > > function 1 places returned marshaled data in resp.body and creates the > > resp.header > > When I was thinking IO bound I was thinking that it was the browser > > actually getting the response data over http. > > Perhaps my IO bound issues are in the creation of the objects or the > > marshaling the response > > to explain function **1 and core. **core is a core transaction module, > > functions 1-3 are types of transactions expense, revenue, budget hence > > that division in the logic. > > On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie <[1]ken at mack-z.com> > wrote: > > > > (method 'poll' of 'select.poll objects) seems to be the place where I > am > > hitting the performance bottleneck. > > To me I could think of that as either the ORM or the previous indexing > > suggestion or both. > > > > References > > > > Visible links > > 1. mailto:ken at mack-z.com > > > -------------- next part -------------- OK so here is what we got so far. Code: I moved from the ORM model to using the expression language.** Also skipped the DictBundle but my marshaling statement is still along the lines of** return json.dumps([dict(r) for r in records] default alchemyencode) Something like that.** No discernible performance improvement, odd. cProfile shows the biggest time spent in the fetchone process iterating through I am guessing in the above statement. So watching CentOS box we barely tick above 12% cpu for this Then we get to Windows. So I click a refresh on the resource. Immediately CPU goes up to 40%, network traffic matches, and they stay at that level for the main duration of the run. But what about memory.** Well the VM in question has a ll of 2gb of ram and frankly it is at 95% baseline. I think we are getting to the problem here. So the sql server side is working for all but the last 2 seconds of the request return, and the whole time memory is basically maxed.** So that might be the issue... On Tue, Jun 6, 2017 at 10:31 PM, David Handy <[1]david at handysoftware.com> wrote: ** **By the way, your I/O waiting could be on database sockets as well as on ** **communication with the browser. ** **On Tuesday, June 6, 2017 10:28pm, "David Handy" <[2]david at handysoftware.com> ** **said: ** **Hi Ken - ** **Your previous email said your web server is running on CentOS and your ** **database is MS SQL Server Express (running on a Windows server, ** **presumably.) ** **What does the CPU usage look like on your web server and database server, ** **respectively, during a test? (It would probably help to have a test where ** **you do several consecutive requests, to make CPU usage more obvious.) ** **David H ** **On Tuesday, June 6, 2017 5:31pm, "Ken MacKenzie" <[3]ken at mack-z.com> said: ** **> _______________________________________________ ** **> TriZPUG mailing list ** **> [4]TriZPUG at python.org ** **> [5]https://mail.python.org/mailman/listinfo/trizpug ** **> [6]http://tripython.org is the Triangle Python Users Group ** **> Wait brain fart, that actually goes back to the IO bound issue. ** **> ** **> Ok so let me think about this. If it is IO bound why does it still show ** **> near 20 seconds for TTFB? ** **> ** **> steps, rough outline ** **> ** **> take request ** **> based on route send request to function 1 ** **> function 1 adds some gravy then redirects to a core function ** **> core function processes the query string in the request and... ** **> does the ORM filter and groub_by setup ** **> handles setting up the DictBundle ** **> for now also deals with date/time and decimal conversion on fields ** **> gets the data ** **> after all the data is back marshals it for the return (json by default ** **> but there is also a csv option) ** **> returns to function 1 ** **> function 1 places returned marshaled data in resp.body and creates the ** **> resp.header ** **> ** **> When I was thinking IO bound I was thinking that it was the browser ** **> actually getting the response data over http. ** **> ** **> Perhaps my IO bound issues are in the creation of the objects or the ** **> marshaling the response ** **> ** **> to explain function 1 and core. core is a core transaction module, ** **> functions 1-3 are types of transactions expense, revenue, budget hence ** **that ** **> division in the logic. ** **> ** **> ** **> On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie <[7]ken at mack-z.com> wrote: ** **> ** **> > ** **> > (method 'poll' of 'select.poll objects) seems to be the place where I ** **am ** **> > hitting the performance bottleneck. ** **> > ** **> > To me I could think of that as either the ORM or the previous indexing ** **> > suggestion or both. ** **> > ** **> > ** **> > ** **> Wait brain fart, that actually goes back to the IO bound issue. ** **> Ok so let me think about this.** If it is IO bound why does it still ** **show ** **> near 20 seconds for TTFB? ** **> steps, rough outline ** **> take request ** **> based on route send request to function 1 ** **> function 1 adds some gravy then redirects to a core function ** **> core function processes the query string in the request and... ** **> ** ** does the ORM filter and groub_by setup ** **> ** ** handles setting up the DictBundle ** **> ** ** for now also deals with date/time and decimal conversion on fields ** **> ** ** gets the data ** **> ** ** after all the data is back marshals it for the return (json by ** **> default but there is also a csv option) ** **> ** ** returns to function 1 ** **> function 1 places returned marshaled data in resp.body and creates the ** **> resp.header ** **> When I was thinking IO bound I was thinking that it was the browser ** **> actually getting the response data over http. ** **> Perhaps my IO bound issues are in the creation of the objects or the ** **> marshaling the response ** **> to explain function **1 and core. **core is a core transaction module, ** **> functions 1-3 are types of transactions expense, revenue, budget hence ** **> that division in the logic. ** **> On Tue, Jun 6, 2017 at 5:22 PM, Ken MacKenzie <[1][8]ken at mack-z.com> wrote: ** **> ** **> (method 'poll' of 'select.poll objects) seems to be the place where I am ** **> hitting the performance bottleneck. ** **> To me I could think of that as either the ORM or the previous indexing ** **> suggestion or both. ** **> ** **> References ** **> ** **> Visible links ** **> 1. mailto:[9]ken at mack-z.com ** **> References Visible links 1. mailto:david at handysoftware.com 2. mailto:david at handysoftware.com 3. mailto:ken at mack-z.com 4. mailto:TriZPUG at python.org 5. https://mail.python.org/mailman/listinfo/trizpug 6. http://tripython.org/ 7. mailto:ken at mack-z.com 8. mailto:ken at mack-z.com 9. mailto:ken at mack-z.com From jeremyhwllc at gmail.com Thu Jun 8 13:38:01 2017 From: jeremyhwllc at gmail.com (Jeremy Davis) Date: Thu, 8 Jun 2017 13:38:01 -0400 Subject: [TriPython] TriLUG Tonight! TARPN Off-The-Grid Text Messaging Message-ID: TriPython, You are invited to join TriLUG tonight! Over the past year we received multiple requests for Software Defined Radio (SDR), Raspberry Pi (RPi), and Internet of Things (IOT) related topics. Tonight we have an excellent topic that offers you a unique educational opportunity to develop skill sets which will cascade directly into your SDR, RPi, and IOT abilities. Tonight you will learn about packet networking over ham radio. Based on interest, you may also have the opportunity to get hands on experience at a following TriLUG hack night. Plans are also in the making to provide ham operator training sessions and licensing exams at our TriLUG Hack Night location. With Linux, an RPi, an antenna, and a Ham license, your devices can communicate long range. This capability may be useful to control a device in your back yard or to retrieve data from a weather balloon carrying a 3d Printed TriTux penguin launched into nearspace during a solar eclipse. Tonight's topic is a highly educational opportunity that will enable you to do so many great things. Tadd Torberg is a wealth of knowledge. You are sure to enjoy this meeting. We will see you there and feel free to invite others to enjoy free pizza, free drinks, free software, (even free beer at the same location 430pm-630pm), and most of all people who care about freedom. Full meeting details https://trilug.org/TARPN-Off-The-Grid-Text-Messaging Jeremy Davis TriLUG PR -------------- next part -------------- TriPython, You are invited to join TriLUG tonight! Over the past year we received multiple requests for Software Defined Radio (SDR), Raspberry Pi (RPi), and Internet of Things (IOT) related topics. Tonight we have an excellent topic that offers you a unique educational opportunity to develop skill sets which will cascade directly into your SDR, RPi, and IOT abilities. Tonight you will learn about packet networking over ham radio. Based on interest, you may also have the opportunity to get hands on experience at a following TriLUG hack night. Plans are also in the making to provide ham operator training sessions and licensing exams at our TriLUG Hack Night location. With Linux, an RPi, an antenna, and a Ham license, your devices can communicate long range. This capability may be useful to control a device in your back yard or to retrieve data from a weather balloon carrying a 3d Printed TriTux penguin launched into nearspace during a solar eclipse. Tonight's topic is a highly educational opportunity that will enable you to do so many great things. Tadd Torberg is a wealth of knowledge. You are sure to enjoy this meeting. We will see you there and feel free to invite others to enjoy free pizza, free drinks, free software, (even free beer at the same location 430pm-630pm), and most of all people who care about freedom. Full meeting details [1]https://trilug.org/TARPN-Off-The-Grid-Text-Messaging Jeremy Davis TriLUG PR References Visible links 1. https://trilug.org/TARPN-Off-The-Grid-Text-Messaging From mlavin at caktusgroup.com Mon Jun 12 09:51:38 2017 From: mlavin at caktusgroup.com (Mark Lavin) Date: Mon, 12 Jun 2017 13:51:38 +0000 Subject: [TriPython] =?utf-8?q?Caktus_Group_is_Hiring_=F0=9F=8C=B5?= Message-ID: Hi TriPythonistas, Caktus Group is looking for a full time Python/Django web developer to join our Durham office. We are accepting applications for entry level (Level I) and intermediate (Level II) applicants. If you enjoy web development and want to work on a variety of exciting projects in a collaborative and team oriented environment then please checkout our the listing on our careers page: https://www.caktusgroup.com/careers/#op-187765-django-web-developer We are also looking to engage with established freelance contractors who want to collaborate with us on upcoming projects. If you work with Django and have part-time to full-time capacity in the near future, let us know and hopefully we can find a fit in some of our incoming work: https://www.caktusgroup.com/careers/#op-173507-django-web-developer-contractor Happy to answer questions you might have about the positions. Next week we are hosting the Durham project night on Monday 6/19 as well as the regular monthly meeting on Thursday 6/22 featuring Phillip Maddux from Signal Sciences talking about his experience with HoneyPy. Both are great opportunities to see the office as well as meet some of the Cakti to find out what it's like to work with us. Hope to hear from you soon. Best, Mark -------------- next part -------------- Hi TriPythonistas, Caktus Group is looking for a full time Python/Django web developer to join our Durham office. We are accepting applications for entry level (Level I) and intermediate (Level II) applicants. If you enjoy web development and want to work on a variety of exciting projects in a collaborative and team oriented environment then please checkout our the listing on our careers page:**[1]https://www.caktusgroup.com/careers/#op-187765-django-web-developer We are also looking to engage with established freelance contractors who want to collaborate with us on upcoming projects. If you work with Django and have part-time to full-time capacity in the near future, let us know and hopefully we can find a fit in some of our incoming work:**[2]https://www.caktusgroup.com/careers/#op-173507-django-web-developer-contractor Happy to answer questions you might have about the positions. Next week**we are hosting the Durham project night on Monday 6/19 as well as the regular monthly meeting on Thursday 6/22 featuring**Phillip Maddux from**Signal Sciences talking about his experience with HoneyPy. Both are great opportunities to see the office as well as meet some of the Cakti to find out what it's like to work with us. Hope to hear from you soon. Best, Mark References Visible links 1. https://www.caktusgroup.com/careers/#op-187765-django-web-developer 2. https://www.caktusgroup.com/careers/#op-173507-django-web-developer-contractor From cbc at unc.edu Tue Jun 13 12:00:15 2017 From: cbc at unc.edu (Calloway, Chris) Date: Tue, 13 Jun 2017 16:00:15 +0000 Subject: [TriPython] Reminder: Chapel Hill Project Night Message-ID: <5BD81FC7-7B06-4B48-BEE7-C2DA783E6F8A@unc.edu> See you tomorrow evening at Chapel Hill Project Night. We will be joined again by the Triangle Deep Learning Study Group. You can join them if that?s your thing. Or you can participate in normal project night activities outside of a study group. Pizza will be provided. Bring your laptop. http://tripython.org/Members/cbc/june-17-chpn When: June 14, 6-9pm Where: Renaissance Computing Institute (RENCI) Biltmore Conference Room, 5th Floor, Europa Center 100 Europa Drive, Suite 590 Chapel Hill What: Chapel Hill Project Night meets on second Wednesdays. Have a project you want to show off, share, seek help with, or just get some work done surrounded by like minded Python lovers? Join us for our monthly project night and do just that! Don't have something to work on? Just need some help with Python? Show up and enjoy the energy, sprint on an open source project, find something interesting to contribute to or be inspired by! The setting is informal and there is no schedule, so don't worry if you show up past the start time. Whether you are a Python newbie needing help or have an open source project you want to share, come hang out and hack. We will be joined again this month by the Triangle Deep Learning Study Group. Plenty of free after hours parking is available in the RENCI parking deck. Bring your laptop. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 -------------- next part -------------- See you tomorrow evening at Chapel Hill Project Night. We will be joined again by the Triangle Deep Learning Study Group. You can join them if that's your thing. Or you can participate in normal project night activities outside of a study group. Pizza will be provided. Bring your laptop. http://tripython.org/Members/cbc/june-17-chpn When: June 14, 6-9pm Where: Renaissance Computing Institute (RENCI) Biltmore Conference Room, 5th Floor, Europa Center 100 Europa Drive, Suite 590 Chapel Hill What: Chapel Hill Project Night meets on second Wednesdays. Have a project you want to show off, share, seek help with, or just get some work done surrounded by like minded Python lovers? Join us for our monthly project night and do just that! Don't have something to work on? Just need some help with Python? Show up and enjoy the energy, sprint on an open source project, find something interesting to contribute to or be inspired by! The setting is informal and there is no schedule, so don't worry if you show up past the start time. Whether you are a Python newbie needing help or have an open source project you want to share, come hang out and hack. We will be joined again this month by the Triangle Deep Learning Study Group. Plenty of free after hours parking is available in the RENCI parking deck. Bring your laptop. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 From cbc at unc.edu Thu Jun 15 09:36:45 2017 From: cbc at unc.edu (Calloway, Chris) Date: Thu, 15 Jun 2017 13:36:45 +0000 Subject: [TriPython] Status of Project Nights in July Message-ID: <7689B1C2-5A33-4CDC-851C-95D085D77B94@unc.edu> Just a heads up for your planning? There will be no project night in Raleigh in July. First Tuesday falls on July 4th and WebAssign will be closed. There will be no project night in Chapel Hill in July. Second Wednesday is during SciPy, where your host will be, and there is a conflict with the room scheduling on that evening necessitating a temporary move down one floor to the Institute for the Environment. I think it is more that I should ask a substitute host, who would have to be a RENCI employee, to manage the logistics of such a room change. Both Raleigh and Chapel Hill Project Nights will resume in August on the usual schedule. There will still be, to the best of my current knowledge, a Durham Project Night on July 17, the third Monday (in addition to the Durham Project Night on this coming Monday June 19). The June and July featured speaker meetings will occur as previously scheduled. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 -------------- next part -------------- Just a heads up for your planning... There will be no project night in Raleigh in July. First Tuesday falls on July 4^th and WebAssign will be closed. There will be no project night in Chapel Hill in July. Second Wednesday is during SciPy, where your host will be, and there is a conflict with the room scheduling on that evening necessitating a temporary move down one floor to the Institute for the Environment. I think it is more that I should ask a substitute host, who would have to be a RENCI employee, to manage the logistics of such a room change. Both Raleigh and Chapel Hill Project Nights will resume in August on the usual schedule. There will still be, to the best of my current knowledge, a Durham Project Night on July 17, the third Monday (in addition to the Durham Project Night on this coming Monday June 19). The June and July featured speaker meetings will occur as previously scheduled. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 From cbc at unc.edu Tue Jun 20 16:15:45 2017 From: cbc at unc.edu (Calloway, Chris) Date: Tue, 20 Jun 2017 20:15:45 +0000 Subject: [TriPython] TriPython June 2017 Meeting: HoneyPy Message-ID: <2754574A-40BA-41B8-9EDA-1381650F2115@unc.edu> Reminder that we have a featured speaker meeting this week at Caktus (see below). Come hear about honey pots and HoneyPy. See you there. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 On 5/30/17, 4:07 PM, "TriZPUG on behalf of Calloway, Chris" wrote: http://tripython.org/Members/cbc/june-17-mtg When: Thursday, June 22, 7pm Where: Caktus Group, 108 Morris St., Durham What: This meeting will provide a light introduction to honeypots and their benefits, and highlight two projects HoneyPy and HoneyDB. Operating honeypot sensors on your internal network is a simple way to make your network ?noisy? and can trip up malicious actors that have already penetrated your network. Also, leveraging data from honeypot sensors on the Internet can be a useful source of threat information. Are you leveraging honeypots in your organization? If not, why not? Adoption of these tools is likely about having simple easy to use interfaces and integrations into existing tools used by an organization. The speaker is Phillip Maddux, a Senior Solutions Engineer at Signal Sciences. Phillip has over 10 years of experience in information security, with the majority of that time focused on application security in the financial services sector. In his spare moments Philip converting ideas to code and committing them to Github. Extemporaneous "lightning talks" of 5-10 minute duration are also welcome and don't need to be pre-announced. Park in the municipal deck on the other side of the Arts Council across W. Morgan St. The meeting will be followed by our usual after-meeting at a nearby tavern for food and beverage. Come join us for a fun and informative evening. From cbc at unc.edu Fri Jun 23 12:29:08 2017 From: cbc at unc.edu (Calloway, Chris) Date: Fri, 23 Jun 2017 16:29:08 +0000 Subject: [TriPython] TriPython June 2017 Meeting: HoneyPy In-Reply-To: <2754574A-40BA-41B8-9EDA-1381650F2115@unc.edu> References: <2754574A-40BA-41B8-9EDA-1381650F2115@unc.edu> Message-ID: <02A7F9F9-37B2-4EF9-BA8E-1FADE734652E@unc.edu> I?d like to thank everyone who made a great meeting last night. Thanks to Caktus Group for the lavish snacks. Thanks to Jeff Bradberry for minding the door and taking care of last minute details. Thanks to Mark Hutchinson for all the publicity work. Thanks to our lightning talkers, Dana, Jeff, Stacy, and Mark. And a big thanks to Phillip Maddux for a very well prepared feature talk and for the laptop camera security shields from Signal Sciences. Phillip?s shared his slides at https://drive.google.com/file/d/0B713haJ0VGmpaDdQOUxNWnZ0bmc/view?usp=sharing and a copy of them is also linked on the TriPython meetings page. There won?t be another meeting until the Monday July 17 Durham project night at Caktus. No Raleigh or Chapel Hill projects nights for July. But all project nights will resume in August. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 On 6/20/17, 4:15 PM, "TriZPUG on behalf of Calloway, Chris" wrote: Reminder that we have a featured speaker meeting this week at Caktus (see below). Come hear about honey pots and HoneyPy. See you there. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 On 5/30/17, 4:07 PM, "TriZPUG on behalf of Calloway, Chris" wrote: http://tripython.org/Members/cbc/june-17-mtg When: Thursday, June 22, 7pm Where: Caktus Group, 108 Morris St., Durham What: This meeting will provide a light introduction to honeypots and their benefits, and highlight two projects HoneyPy and HoneyDB. Operating honeypot sensors on your internal network is a simple way to make your network ?noisy? and can trip up malicious actors that have already penetrated your network. Also, leveraging data from honeypot sensors on the Internet can be a useful source of threat information. Are you leveraging honeypots in your organization? If not, why not? Adoption of these tools is likely about having simple easy to use interfaces and integrations into existing tools used by an organization. The speaker is Phillip Maddux, a Senior Solutions Engineer at Signal Sciences. Phillip has over 10 years of experience in information security, with the majority of that time focused on application security in the financial services sector. In his spare moments Philip converting ideas to code and committing them to Github. Extemporaneous "lightning talks" of 5-10 minute duration are also welcome and don't need to be pre-announced. Park in the municipal deck on the other side of the Arts Council across W. Morgan St. The meeting will be followed by our usual after-meeting at a nearby tavern for food and beverage. Come join us for a fun and informative evening. _______________________________________________ TriZPUG mailing list TriZPUG at python.org https://mail.python.org/mailman/listinfo/trizpug http://tripython.org is the Triangle Python Users Group From cbc at unc.edu Fri Jun 23 12:47:02 2017 From: cbc at unc.edu (Calloway, Chris) Date: Fri, 23 Jun 2017 16:47:02 +0000 Subject: [TriPython] TriPython July 2017 Meeting: Just What Is A Quality Engineer? Message-ID: <0123DB8E-14E7-43DE-9E33-DDAA92AC3F84@unc.edu> http://tripython.org/Members/cbc/july-17-mtg When: Thursday, July 27, 7pm Where: Renaissance Computing Institute (RENCI) Biltmore Conference Room, 5th Floor 100 Europa Drive, Suite 590 Chapel Hill What: A great plurality of companies today, who deal with the delivery of software or providing software services should have a team responsible for checking the quality of their products before they get into their customers? hands. This team, far too often, is comprised of a single person, usually labeled with the term ?QA?, and is responsible for making sure that all known issues with the product are verified and that some level of testing is performed before it can be ?shipped? to customers. Not everyone really knows what happens during this phase, but it is well-known that someone, somehow, installs, configures and ?tests? all the facets and features of a product before it ?goes out the door?. The stereotype is that somewhere in the building where you work, there is at least one person, ?not suitable? to be a developer, who is relegated to pushing buttons and clicking elements on web ui elements to make sure that things work as advertised. These are the ?button pushers?. If you have some of this species inhabiting your work environment, good for you. But if you really want to deliver quality with your products, then what you need is a Quality Engineer, a rare hybrid species that merges software development, forensics, DEVOPS and creativity skills into a super being. Og Maciel presents. Og is a Senior Manager of Quality Engineering for the Red Hat Satellite team. He has spent the last 5+ years building a team of black belt quality engineers responsible for the automation of complex systems and delivering quality products through the use of continuous delivery of processes. He is also a podcaster, a dad, and an avid reader. Extemporaneous "lightning talks" of 5-10 minute duration are also welcome and don't need to be pre-announced. Lightning talks are for you to "show and tell" something you've learned about Python recently, no matter how small. We all use Python, therefore, we are always learning something new about Python that we can tell others. Plenty of free parking is available in the RENCI parking deck. The meeting will be followed by our usual after-meeting at a nearby tavern for food and beverage. Come join us for a fun and informative evening. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 -------------- next part -------------- [1]http://tripython.org/Members/cbc/july-17-mtg When: Thursday, July 27, 7pm Where: Renaissance Computing Institute (RENCI) Biltmore Conference Room, 5th Floor 100 Europa Drive, Suite 590 Chapel Hill What: A great plurality of companies today, who deal with the delivery of software or providing software services should have a team responsible for checking the quality of their products before they get into their customers' hands. This team, far too often, is comprised of a single person, usually labeled with the term "QA", and is responsible for making sure that all known issues with the product are verified and that some level of testing is performed before it can be `shipped' to customers. Not everyone really knows what happens during this phase, but it is well-known that someone, somehow, installs, configures and `tests' all the facets and features of a product before it `goes out the door'. The stereotype is that somewhere in the building where you work, there is at least one person, "not suitable" to be a developer, who is relegated to pushing buttons and clicking elements on web ui elements to make sure that things work as advertised. These are the `button pushers'. If you have some of this species inhabiting your work environment, good for you. But if you really want to deliver quality with your products, then what you need is a Quality Engineer, a rare hybrid species that merges software development, forensics, DEVOPS and creativity skills into a super being. Og Maciel presents. Og is a Senior Manager of Quality Engineering for the Red Hat Satellite team. He has spent the last 5+ years building a team of black belt quality engineers responsible for the automation of complex systems and delivering quality products through the use of continuous delivery of processes. He is also a podcaster, a dad, and an avid reader. Extemporaneous "lightning talks" of 5-10 minute duration are also welcome and don't need to be pre-announced. Lightning talks are for you to "show and tell" something you've learned about Python recently, no matter how small. We all use Python, therefore, we are always learning something new about Python that we can tell others. Plenty of free parking is available in the RENCI parking deck. The meeting will be followed by our usual after-meeting at a nearby tavern for food and beverage. Come join us for a fun and informative evening. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 References Visible links 1. http://tripython.org/Members/cbc/july-17-mtg From omaciel at ogmaciel.com Sat Jun 24 15:59:12 2017 From: omaciel at ogmaciel.com (Og Maciel) Date: Sat, 24 Jun 2017 15:59:12 -0400 Subject: [TriPython] Interview with Dan Bader Message-ID: Howdy! This last Wednesday, June 21st, together with 2 fellow Red Hatters/podcasters, I had the opportunity to interview Dan Bader, a self proclaimed ?Pythonista nut? and author of the book ?Python Tricks. The interview was done live via Youtube with the participation of some of our listeners and I figured I?d share the link to the video with you guys: * https://www.youtube.com/watch?v=uZ-IrwCKGFg We do a live episode every Wednesday which then gets published in audio format for our podcast. All but a few of our 106 episodes have been done in Brazilian Portuguese, but we?re trying to do more episodes in English. We did interview Kenneth Reitz (http://castalio.info/episodio-80-kenneth-reitz-python-requests.html) and Tarek Ziade (http://castalio.info/episodio-83-tarek-ziade-mozilla.html) a while back and are looking to interview a Python core developer fairly soon. Anyhow, hope you guys like the video/interview. -- Og Maciel - GPG Keys: CD03D583 omaciel at ogmaciel.com https://omaciel.github.io From lisa at renci.org Tue Jun 27 10:48:05 2017 From: lisa at renci.org (Lisa Stillwell) Date: Tue, 27 Jun 2017 14:48:05 +0000 Subject: [TriPython] TriPython July 2017 Meeting: Just What Is A Quality Engineer? In-Reply-To: <0123DB8E-14E7-43DE-9E33-DDAA92AC3F84@unc.edu> References: <0123DB8E-14E7-43DE-9E33-DDAA92AC3F84@unc.edu> Message-ID: <4887A531-0AE7-4D38-AC8F-C0EDA7A6C13E@renci.org> Tony, Thought you might be interested in attending this: > On Jun 23, 2017, at 12:47 PM, Calloway, Chris wrote: > > [1]http://tripython.org/Members/cbc/july-17-mtg > > > > When: Thursday, July 27, 7pm > > Where: Renaissance Computing Institute (RENCI) > > Biltmore Conference Room, 5th Floor > > 100 Europa Drive, Suite 590 > > Chapel Hill > > What: A great plurality of companies today, who deal with the delivery of > software or providing software services should have a team responsible for > checking the quality of their products before they get into their > customers' hands. This team, far too often, is comprised of a single > person, usually labeled with the term "QA", and is responsible for making > sure that all known issues with the product are verified and that some > level of testing is performed before it can be `shipped' to customers. Not > everyone really knows what happens during this phase, but it is well-known > that someone, somehow, installs, configures and `tests' all the facets and > features of a product before it `goes out the door'. The stereotype is > that somewhere in the building where you work, there is at least one > person, "not suitable" to be a developer, who is relegated to pushing > buttons and clicking elements on web ui elements to make sure that things > work as advertised. These are the `button pushers'. If you have some of > this species inhabiting your work environment, good for you. But if you > really want to deliver quality with your products, then what you need is a > Quality Engineer, a rare hybrid species that merges software development, > forensics, DEVOPS and creativity skills into a super being. Og Maciel > presents. Og is a Senior Manager of Quality Engineering for the Red Hat > Satellite team. He has spent the last 5+ years building a team of black > belt quality engineers responsible for the automation of complex systems > and delivering quality products through the use of continuous delivery of > processes. He is also a podcaster, a dad, and an avid reader. > Extemporaneous "lightning talks" of 5-10 minute duration are also welcome > and don't need to be pre-announced. Lightning talks are for you to "show > and tell" something you've learned about Python recently, no matter how > small. We all use Python, therefore, we are always learning something new > about Python that we can tell others. Plenty of free parking is available > in the RENCI parking deck. The meeting will be followed by our usual > after-meeting at a nearby tavern for food and beverage. Come join us for a > fun and informative evening. > > > > -- > > Sincerely, > > > > Chris Calloway > > Applications Analyst > > University of North Carolina > > Renaissance Computing Institute > > (919) 599-3530 > > > > References > > Visible links > 1. http://tripython.org/Members/cbc/july-17-mtg > _______________________________________________ > TriZPUG mailing list > TriZPUG at python.org > https://mail.python.org/mailman/listinfo/trizpug > http://tripython.org is the Triangle Python Users Group From lisa at renci.org Tue Jun 27 10:53:05 2017 From: lisa at renci.org (Lisa Stillwell) Date: Tue, 27 Jun 2017 14:53:05 +0000 Subject: [TriPython] TriPython July 2017 Meeting: Just What Is A Quality Engineer? In-Reply-To: <4887A531-0AE7-4D38-AC8F-C0EDA7A6C13E@renci.org> References: <0123DB8E-14E7-43DE-9E33-DDAA92AC3F84@unc.edu> <4887A531-0AE7-4D38-AC8F-C0EDA7A6C13E@renci.org> Message-ID: <995682EF-547F-45E9-9438-3AE13E0E3019@renci.org> Sorry fellow TriPythoners - I just came back from vacation and I am obviously still a bit dreamy ? On Jun 27, 2017, at 10:48 AM, Lisa Stillwell > wrote: Tony, Thought you might be interested in attending this: On Jun 23, 2017, at 12:47 PM, Calloway, Chris > wrote: [1]http://tripython.org/Members/cbc/july-17-mtg When: Thursday, July 27, 7pm Where: Renaissance Computing Institute (RENCI) Biltmore Conference Room, 5th Floor 100 Europa Drive, Suite 590 Chapel Hill What: A great plurality of companies today, who deal with the delivery of software or providing software services should have a team responsible for checking the quality of their products before they get into their customers' hands. This team, far too often, is comprised of a single person, usually labeled with the term "QA", and is responsible for making sure that all known issues with the product are verified and that some level of testing is performed before it can be `shipped' to customers. Not everyone really knows what happens during this phase, but it is well-known that someone, somehow, installs, configures and `tests' all the facets and features of a product before it `goes out the door'. The stereotype is that somewhere in the building where you work, there is at least one person, "not suitable" to be a developer, who is relegated to pushing buttons and clicking elements on web ui elements to make sure that things work as advertised. These are the `button pushers'. If you have some of this species inhabiting your work environment, good for you. But if you really want to deliver quality with your products, then what you need is a Quality Engineer, a rare hybrid species that merges software development, forensics, DEVOPS and creativity skills into a super being. Og Maciel presents. Og is a Senior Manager of Quality Engineering for the Red Hat Satellite team. He has spent the last 5+ years building a team of black belt quality engineers responsible for the automation of complex systems and delivering quality products through the use of continuous delivery of processes. He is also a podcaster, a dad, and an avid reader. Extemporaneous "lightning talks" of 5-10 minute duration are also welcome and don't need to be pre-announced. Lightning talks are for you to "show and tell" something you've learned about Python recently, no matter how small. We all use Python, therefore, we are always learning something new about Python that we can tell others. Plenty of free parking is available in the RENCI parking deck. The meeting will be followed by our usual after-meeting at a nearby tavern for food and beverage. Come join us for a fun and informative evening. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 References Visible links 1. http://tripython.org/Members/cbc/july-17-mtg _______________________________________________ TriZPUG mailing list TriZPUG at python.org https://mail.python.org/mailman/listinfo/trizpug http://tripython.org is the Triangle Python Users Group -------------- next part -------------- Sorry fellow TriPythoners - I just came back from vacation and I am obviously still a bit dreamy **** On Jun 27, 2017, at 10:48 AM, Lisa Stillwell <[1]lisa at renci.org> wrote: Tony, Thought you might be interested in attending this: On Jun 23, 2017, at 12:47 PM, Calloway, Chris <[2]cbc at unc.edu> wrote: [1][3]http://tripython.org/Members/cbc/july-17-mtg When: Thursday, July 27, 7pm Where: Renaissance Computing Institute (RENCI) Biltmore Conference Room, 5th Floor 100 Europa Drive, Suite 590 Chapel Hill What: A great plurality of companies today, who deal with the delivery of software or providing software services should have a team responsible for checking the quality of their products before they get into their customers' hands. This team, far too often, is comprised of a single person, usually labeled with the term "QA", and is responsible for making sure that all known issues with the product are verified and that some level of testing is performed before it can be `shipped' to customers. Not everyone really knows what happens during this phase, but it is well-known that someone, somehow, installs, configures and `tests' all the facets and features of a product before it `goes out the door'. The stereotype is that somewhere in the building where you work, there is at least one person, "not suitable" to be a developer, who is relegated to pushing buttons and clicking elements on web ui elements to make sure that things work as advertised. These are the `button pushers'. If you have some of this species inhabiting your work environment, good for you. But if you really want to deliver quality with your products, then what you need is a Quality Engineer, a rare hybrid species that merges software development, forensics, DEVOPS and creativity skills into a super being. Og Maciel presents. Og is a Senior Manager of Quality Engineering for the Red Hat Satellite team. He has spent the last 5+ years building a team of black belt quality engineers responsible for the automation of complex systems and delivering quality products through the use of continuous delivery of processes. He is also a podcaster, a dad, and an avid reader. Extemporaneous "lightning talks" of 5-10 minute duration are also welcome and don't need to be pre-announced. Lightning talks are for you to "show and tell" something you've learned about Python recently, no matter how small. We all use Python, therefore, we are always learning something new about Python that we can tell others. Plenty of free parking is available in the RENCI parking deck. The meeting will be followed by our usual after-meeting at a nearby tavern for food and beverage. Come join us for a fun and informative evening. -- Sincerely, Chris Calloway Applications Analyst University of North Carolina Renaissance Computing Institute (919) 599-3530 References Visible links 1. [4]http://tripython.org/Members/cbc/july-17-mtg _______________________________________________ TriZPUG mailing list [5]TriZPUG at python.org https://mail.python.org/mailman/listinfo/trizpug http://tripython.org is the Triangle Python Users Group References Visible links 1. mailto:lisa at renci.org 2. mailto:cbc at unc.edu 3. http://tripython.org/Members/cbc/july-17-mtg 4. http://tripython.org/Members/cbc/july-17-mtg 5. mailto:TriZPUG at python.org