From jep200404 at columbus.rr.com Fri Sep 1 16:21:33 2017 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Fri, 1 Sep 2017 16:21:33 -0400 Subject: [CentralOH] Captions for Pyvideo Videos from IBM speech to text transcription In-Reply-To: References: Message-ID: <20170901162133.67c74899.jep200404@columbus.rr.com> On Wed, 30 Aug 2017 15:54:12 -0400 (EDT), R P Herrold wrote: > speech to text > https://console.bluemix.net/docs/openwhisk/openwhisk_watson_speechtotext.html#converting-speech-to-text Imagine this used to create captions for videos on pyvideo.org. From jep200404 at columbus.rr.com Fri Sep 1 23:12:44 2017 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Fri, 1 Sep 2017 23:12:44 -0400 Subject: [CentralOH] =?utf-8?q?2017-08-31_=E9=81=93=E5=A0=B4_Scribbles_?= =?utf-8?b?76SY5pu4L+aDoeaWhz8gcHlyYW1pZCBzbGlkaW5nIHYgZXVsZXIgIzE4OyBi?= =?utf-8?q?ugs?= Message-ID: <20170901231244.47dc0a56.jep200404@columbus.rr.com> 8 folks tonight ubuntu.com is good distro for beginners wp:VirtualBox wp:Hyper-V wp: prefix means Wikipedia To get good answers, consider following the advice in the links below. http://catb.org/~esr/faqs/smart-questions.html http://web.archive.org/web/20090627155454/www.greenend.org.uk/rjk/2000/06/14/quoting.html freecoders.com??? openhash.org https://www.edureka.co/ https://www.reddit.com/r/dailyprogrammer/ Compare: https://www.reddit.com/r/dailyprogrammer/comments/6vi9ro/170823_challenge_328_intermediate_pyramid_sliding/ http://nbviewer.jupyter.org/github/james-prior/euler/blob/master/euler-018-maximum-path-sum-i-20161225.ipynb i wonder about other ways of doing cell 7 consolidating body of loop into one ugly long statement recursion http://nbviewer.jupyter.org/github/james-prior/euler/blob/master/euler-015-lattice-paths-20130726.ipynb Compare: Children multiplication tables practice test with Bash script on Linux https://linuxconfig.org/children-multiplication-tables-practice-test-with-bash-script-on-linux http://nbviewer.jupyter.org/github/james-prior/cohpy/blob/master/20170810-dojo-parsing-input-file.ipynb This has multiple bugs, especially cell #3. Today is a Good Day to Learn Python https://www.linux.com/learn/intro-to-Linux/2017/8/today-good-day-learn-python Astroplan Python library makes astronomy research planning easier https://opensource.com/article/17/8/astroplan DeepSPADE (alias DeepSmokey): A Machine-Learning System That Collects Spam from the Internet https://www.linux.com/blog/event/open-source-summit-la/2017/8/deepspade-alias-deepsmokey-machine-learning-system-collects-spam-internet Gab Launches Legal Fund to Take on Silicon Valley Censorship http://www.breitbart.com/tech/2017/08/22/gab-launches-legal-fund-to-take-on-silicon-valley-censorship/ react.js can be used with django Apache Foundation and Facebook in Standoff Over React.js License http://windowsitpro.com/open-source/apache-foundation-and-facebook-standoff-over-reactjs-license 3 open source Python GUI frameworks https://opensource.com/resources/python/gui-frameworks Python articles at opensource.com https://opensource.com/tags/python?src=programming_resource_menu1 Mentoring: Your path to immortality https://opensource.com/article/17/8/mentoring-your-path-immortality From jep200404 at columbus.rr.com Sun Sep 10 21:46:23 2017 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Sun, 10 Sep 2017 21:46:23 -0400 Subject: [CentralOH] =?utf-8?q?2017-09-07_=E9=81=93=E5=A0=B4_Scribbles_?= =?utf-8?b?76SY5pu4L+aDoeaWhz8gcHl2ZW52IHZlcnN1cyB2ZW52IHZlcnN1cyB2aXJ0?= =?utf-8?q?ualenv=3B_github_backups=3B_brandon=27s_adventure=3B_check_writ?= =?utf-8?q?ing?= Message-ID: <20170910214623.63e7b172.jep200404@columbus.rr.com> pyvenv versus venv I have been so confused about pyvenv versus venv for virtual environments for Python3 that I shunned both of them. Also, pyvenv but not venv would work on one distro, and venv but not pyvenv would work on another distro. Now venv works on both distros. Also, "28.3. venv ? Creation of virtual environments"[1] says: Note The pyvenv script has been deprecated as of Python 3.6 in favor of using python3 -m venv ... So "python3 -m venv /path/to/new/virtual/environment" is the way preferred by the Python Software Foundation. Unfortunately, venv is not happy on a Centos 7 installation: [doj at sbc ~]$ python3 Python 3.4.5 (default, Nov 9 2016, 16:24:59) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> [doj at sbc ~]$ python3 -m venv mobyvenv Error: Command '['/home/doj/mobyvenv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1 [doj at sbc ~]$ so I resort to something like: virtualenv env -p `which python3` systemd is the best example of Suck. http://suckless.org/sucks/systemd https://twitter.com/systemdsucks not http://systemdsucks.com/ other examples of suck http://newromesucks.com/ Thanks again to Jamie Mueller for his efforts in ending New Rome. wp:New Rome, Ohio http://www.newromesucks.com/topten.html wp:Brice, Ohio http://www.dispatch.com/content/stories/editorials/2015/11/28/1-brice-finds-another-way.html http://abc6onyourside.com/news/local/on-your-side-speed-camera-inside-orange-barrel-gets-attention-but-its-legal backing up github issues How to Download GitHub Issues from the Command Line https://adriansieber.com/download-github-issues-from-command-line/ doj at sbc:~$ for i in `seq 124 -1 1`; do echo $i;if wget -x -a log https://github.com/james-prior/cohpy/pull/$i; then :; else wget -x -a log https://github.com/james-prior/cohpy/issue/$i;fi;sleep 180;done 3 consequences of coding in the open https://opensource.com/open-organization/17/8/transparency-at-reaction A programmer?s cleaning guide for messy sensor data https://opensource.com/article/17/9/messy-sensor-data a rival language Must go faster, must go faster! Oracle lobs Java EE into GitHub, vows rapid Java SE releases http://www.theregister.co.uk/2017/09/06/oracle_java_ee_java_se_github/ [2017-09-06] Challenge #330 [Intermediate] Check Writer https://www.reddit.com/r/dailyprogrammer/comments/6yep7x/20170906_challenge_330_intermediate_check_writer/ Heirloom Software: the Past as Adventure http://www.linuxjournal.com/content/heirloom-software-past-adventure Brandon Rhodes' Adventure in Python3 http://pyvideo.org/pyohio-2011/pyohio-2011--sunday-lightning-talks.html http://rhodesmill.org/brandon/2012/adventure/ https://github.com/brandon-rhodes/python-adventure/tree/master/adventure [1] 28.3. venv ? Creation of virtual environments https://docs.python.org/3/library/venv.html From miller.eric.t at gmail.com Fri Sep 15 16:06:49 2017 From: miller.eric.t at gmail.com (Eric Miller) Date: Fri, 15 Sep 2017 16:06:49 -0400 Subject: [CentralOH] PyPI security issue Message-ID: SK-CSIRT identified malicious software libraries PyPI, posing as well known libraries. https://news.ycombinator.com/item?id=15256121 http://www.nbu.gov.sk/skcsirt-sa-20170909-pypi/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at intellovations.com Mon Sep 18 19:44:53 2017 From: eric at intellovations.com (Eric Floehr) Date: Mon, 18 Sep 2017 19:44:53 -0400 Subject: [CentralOH] September Monthly Meeting Message-ID: RSVP Here: https://www.meetup.com/Central-Ohio-Python-Users-Group/events/236712516/ Come and learn, share, grow, meet new people, and visit old friends at our monthly meeting! We'll be talking about the Python programming language and anything that intersects it, and the cool stuff you can do with it. This month we welcome Erik Welch, Senior Software Developer at Continuum Analytics to talk about how IPython and Jupyter Notebooks have changed the way we develop, interact with, and share code in Python. He will talk about recent advances in the Jupyter ecosystem, and explore ways to enhance your Jupyter experience. Some specific examples of possible topics covered include Jupyter Notebooks, JupyterLab, JupyterHub, kernels, plugins, extensions, ipywidgets, bqplots, bokeh, dask, and more. Expect plenty of live examples and hacking! Afterwards we'll be heading to Brazenhead on 5th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jep200404 at columbus.rr.com Tue Sep 19 16:26:13 2017 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Tue, 19 Sep 2017 16:26:13 -0400 Subject: [CentralOH] =?utf-8?q?2017-09-14_=E9=81=93=E5=A0=B4_Scribbles_?= =?utf-8?b?76SY5pu4L+aDoeaWhz8gUG9wdWxhcml0eSwgTG9nZ2luZywgUGFuZGFz?= Message-ID: <20170919162613.7f37faa8.jep200404@columbus.rr.com> wifi was not good tonight Everyone loves programming in Python! You disagree? But it's the fastest growing, says Stack Overflow https://www.theregister.co.uk/2017/09/09/python_is_an_expander_not_a_constrictor/ Oracle prepares to spin off Java EE to Eclipse Foundation http://www.zdnet.com/article/oracle-prepares-to-spin-off-java-ee-to-eclipse-foundation/#ftag=RSSbaffb68 A guide to logging in Python https://opensource.com/article/17/9/python-logging Irma @ CMH: .42 inch total; 9 mph wind equifax apache something exploit? argentine admin admin krebs on security http://www.krebsonsecurity.com/ wp:Brian Krebs wp: prefix means Wikipedia To get good answers, consider following the advice in the links below. http://catb.org/~esr/faqs/smart-questions.html http://web.archive.org/web/20090627155454/www.greenend.org.uk/rjk/2000/06/14/quoting.html Python explosion blamed on pandas Data science fad just won't die https://www.theregister.co.uk/2017/09/14/python_explosion_blamed_on_pandas/ Python "Nightmare" http://news.nationalgeographic.com/news/2009/09/090911-pythons-florida-giant-snakes_2.html Linux fmt command - usage and examples https://www.howtoforge.com/linux-fmt-command/ wp:less (Unix) wp:more (command) wp:MOST Bus wp:CAN bus From pybokeh at gmail.com Thu Sep 21 23:14:10 2017 From: pybokeh at gmail.com (pybokeh) Date: Thu, 21 Sep 2017 23:14:10 -0400 Subject: [CentralOH] Pandas Cheat Sheet Message-ID: Travis at the Python DoJo suggested that I share my Pandas cheat sheet. I've shared it a some years ago, but thought I share it again. nbviewer.jupyter.org/github/pybokeh/jupyter_notebooks/blob/master/pandas/PandasCheatSheet.ipynb - Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From miller.eric.t at gmail.com Fri Sep 22 08:09:57 2017 From: miller.eric.t at gmail.com (Eric Miller) Date: Fri, 22 Sep 2017 08:09:57 -0400 Subject: [CentralOH] Pandas Cheat Sheet In-Reply-To: References: Message-ID: this is awesome, and timely for me as I'm in the middle of the pandas section of Python for Data Science and Machine Learning Bootcamp. Thank you! On Thu, Sep 21, 2017 at 11:14 PM, pybokeh wrote: > Travis at the Python DoJo suggested that I share my Pandas cheat sheet. > I've shared it a some years ago, but thought I share it again. > > nbviewer.jupyter.org/github/pybokeh/jupyter_notebooks/blob/master/pandas/ > PandasCheatSheet.ipynb > > - Daniel > > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > https://mail.python.org/mailman/listinfo/centraloh > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pybokeh at gmail.com Fri Sep 22 08:43:20 2017 From: pybokeh at gmail.com (pybokeh) Date: Fri, 22 Sep 2017 08:43:20 -0400 Subject: [CentralOH] Pandas Cheat Sheet In-Reply-To: References: Message-ID: Glad you like. You may also be interested in Tom Augsperger's blog series on modern pandas. He is major contributor to pandas. https://tomaugspurger.github.io/ On Sep 22, 2017 8:13 AM, "Eric Miller" wrote: > this is awesome, and timely for me as I'm in the middle of the pandas > section of Python for Data Science and Machine Learning Bootcamp. > > Thank you! > > On Thu, Sep 21, 2017 at 11:14 PM, pybokeh wrote: > >> Travis at the Python DoJo suggested that I share my Pandas cheat sheet. >> I've shared it a some years ago, but thought I share it again. >> >> nbviewer.jupyter.org/github/pybokeh/jupyter_notebooks/blob/ >> master/pandas/PandasCheatSheet.ipynb >> >> - Daniel >> >> _______________________________________________ >> CentralOH mailing list >> CentralOH at python.org >> https://mail.python.org/mailman/listinfo/centraloh >> >> > > _______________________________________________ > CentralOH mailing list > CentralOH at python.org > https://mail.python.org/mailman/listinfo/centraloh > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jep200404 at columbus.rr.com Sat Sep 23 15:59:00 2017 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Sat, 23 Sep 2017 15:59:00 -0400 Subject: [CentralOH] =?utf-8?q?2017-09-21_=E9=81=93=E5=A0=B4_Scribbles_?= =?utf-8?b?76SY5pu4L+aDoeaWhz8gT2RvcmFtYTsgc3BhY2VkIHJlcGV0aXRpb24gZmxh?= =?utf-8?q?sh_cards=3B_emergent_design=3B_embedding_C++=3B_why_can=27t_joh?= =?utf-8?q?nny_read?= Message-ID: <20170923155900.5a99467a.jep200404@columbus.rr.com> Refactoring fun: How would you write Scheduler._dynOrder() in the following? https://github.com/dae/anki/blob/bd414595de971ac29b54d521d734140eee7d80ee/anki/sched.py >From "The Mirror Pool" Largo Sanvean wp:Django (web framework) wp:Gypsy jazz wp:Hawthorne effect wp:Elton Mayo wp: prefix means Wikipedia To get good answers, consider following the advice in the links below. http://catb.org/~esr/faqs/smart-questions.html http://web.archive.org/web/20090627155454/www.greenend.org.uk/rjk/2000/06/14/quoting.html Someone is uncomfortable with tdd doing away with design, that emergent design will not always lead to optimal results. Some things just can not be done by committees (or pairs). wp:Emergent_Design#Emergent_design_in_agile_software_development Is Design Dead? https://martinfowler.com/articles/designDead.html I remembered that there was an article about using C++ in embedded applications. It talked about using the things that cause no trouble, and avoiding the things that do cause trouble. I thought it was in Embedded Systems Journal in the mid 2000s and was by Dan Saks. Since he gave a presentation at Columbus Code Camp a few years about about measuring readability of coding styles, I have been looking for that article. I have not found it, but I have found something similar. C++ for Embedded C Programmers Dan Saks http://www.dansaks.com/talks/ESC-205.pdf http://info.pillartechnology.com/kata flash card wp:Anki (software) damien elmes wp:Spaced repetition https://github.com/dae/anki turnoffs: strong code smell: for example, study Finder._order() in https://github.com/dae/anki/blob/bd414595de971ac29b54d521d734140eee7d80ee/anki/find.py yet refactoring not welcome: See https://github.com/dae/anki/blob/bd414595de971ac29b54d521d734140eee7d80ee/README.contributing licensing: contributors must license changes with BSD license, even though code is GNU AGPL, version 3 or later is that for dual licensing for private benefit of Damien? I wonder how Mnemosyne and OpenCards compare. wp:Mnemosyne (software) Python! open source! (hopefully only open source) wp:OpenCards PEP 8 -- Style Guide for Python Code https://www.python.org/dev/peps/pep-0008/ pylint how to have it automatically re-evaluate code when a file changes? (like py.test -f .) Definitive regular expression dox for Python 6.2. re ? Regular expression operations https://docs.python.org/3/library/re.htm learn by reading and experimenting Jupyter notebooks are great for that. Mastering Regular Expressions Third Edition http://regex.info/book.html http://shop.oreilly.com/product/9780596528126.do https://github.com/james-prior/euler https://github.com/james-prior/cohpy https://github.com/cohpy https://www.reddit.com/r/dailyprogrammer/wiki/challenges Why Can't Programmers.. Program? https://blog.codinghorror.com/why-cant-programmers-program/ Separating Programming Sheep from Non-Programming Goats https://blog.codinghorror.com/separating-programming-sheep-from-non-programming-goats/ From jep200404 at columbus.rr.com Tue Sep 26 11:45:56 2017 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Tue, 26 Sep 2017 11:45:56 -0400 Subject: [CentralOH] Robot Meeting Tonight; Autonomous Python Car Message-ID: <20170926114556.21d3eacc.jep200404@columbus.rr.com> robots meeting tonight unescorted soul bags welcome Joint Meetup with VR Columbus: VR demo and robotics racing in the works https://www.meetup.com/AutonomyHub/events/243521906/ DIY autonomous car (that uses Python) Donkey Car http://www.donkeycar.com/ I don't know that a donkey car will be at tonight's meeting. We should have a demo at a Python meeting! Meeting at the Murphy Building reminds me of an eponymous law. From jep200404 at columbus.rr.com Thu Sep 28 16:21:57 2017 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Thu, 28 Sep 2017 16:21:57 -0400 Subject: [CentralOH] =?utf-8?q?2017-09-25_=E6=9C=83=E8=AD=B0_Scribbles_?= =?utf-8?b?76SY5pu4L+aDoeaWhz86IEVyaWsgV2VsY2ggSnVweXRlciBMYWIgRGFzaw==?= Message-ID: <20170928162157.40404fd2.jep200404@columbus.rr.com> Thanks to Christoph Baker and Pillar for their generous prosperity. They gave us plenty of pizza, salad, cookies, and beverages. There was also hummus for the first time. techelevator data analysis has been coming for about a year and a half likes doing everything in python terminal size zach at cohpy:~$ echo $LINES $COLUMNS 26 100 zach at cohpy:~$ grep 'LINES\|COLUMNS' ~/.bashrc # update the values of LINES and COLUMNS. export LINES export COLUMNS zach at cohpy:~$ python3 Python 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.environ['LINES'] '26' >>> os.environ['COLUMNS'] '100' >>> zach at cohpy:~$ xdpyinfo | less one of last years challenges (the graphical one) had discussion of portable way of determining screen size in pixels has background in physics work in dublin now python for ai and neat little projects want something done quickly and don't know how to do it in bash about 32 folks tonight python booth at ohio linux festival 2017-09-30 need to have collection of python books at olf 2017-10-30 monthly meeting: Michael Handler and Leonard embedded train track sensor and did real-time analysis 2017-10-14 columbuscodecamp.com Christoph Baker folks should apply to pillar job interview process Erik Welch https://github.com/eriknw cytoolz dask-patternsearch eqpy benchtoolz metafunc https://twitter.com/eriknwelch works on pytoolz and cytoolz (faster) dask https://dask.pydata.org/en/latest/ dask python3 python2 django pandas jupyter notebook conda dask numba bokeh anaconda holoviews came for the language; stayed for the community JupyterLab human-centered interactive much coolness has a bias for making things fast holoviews jupytercon (was last month in nyc) https://github.com/OpenGeoscience/geonotebook A Jupyter notebook extension for geospatial visualization and analysis nbextensions ExecuteTime zenmode pretty background more full screen hides header code pretty little hammer icon hide header variable inspector ExecuteTime automatically shows execution time for each cell localhost:8888/tree?#nbextensions_configurator %qtconsole partially works for me have to restart kernel after qt window exists https://jupyter.org/qtconsole/stable/ core jupyter team uses conda to build and share packages !conda list !conda install -c conda-forge jupyter jupyter_contrib_nbextensions jupyter-contrib-nbextensions.readthedocs.io/en/latest/ import seaborn as sns df = sns.load_dataset('iris') df # show prettier dataframe # cool two-dimensional plots sns.jointplot(df.sepal_length, df.sepal_width) sns.jointplot(df.sepal_length, df.petal_length) sns.jointplot(df.sepal_length, df.petal_length, kind='kde') from __future__ import print_function, division from sympy import * init_printing() ell_min, ell, ell_max = symbols('ell_min,ell,ell_max', integer=True) summ = summation((2*ell * 1), (ell, ell_min, ell_max)) summ ell_min github.com/OpenGeoscience/geonotebook scroll down for eye candy jupyterhub a "thing explainer" overview EM GeoSci electromagnetics https://em.geosci.xyz/index.html erik likes gnuplot %gnuplot https://github.com/has2k1/gnuplot_kernel/blob/master/examples/gnuplot-kernel.ipynb http://nbviewer.jupyter.org/github/has2k1/gnuplot_kernel/blob/master/examples/gnuplot-kernel.ipynb https://github.com/Calysto/notebook-extensions notebook widgets from ipyleaflet import Map Map(center=[39.975, -82.998], zoom=11) from pythreejs import * Renderer(camera=c, scene=scene, control=[OrbitControls(controlling=c)]) %time funky_func() %timeit funky_func() %%time %%timeit funky_func() %%prun -s cumulative -l 5 funky_func() %load_ext snakeviz %snakeviz funky_func() sunburst versus icicle %load_ext line_profiler %lprun -f funky_func funky_func() sunburst versus icicle %load_ext memory_profiler %memit? %memit funky_func # %mprun -f funky_func funcy_func() # needs to be from a file github.com/has2k1/gnuplot_kernel/blob/master/examples/gnuplotkernel/ipynb Unofficial Jupyter Notebook Extensions https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/ jupyter.org/widgets.html folium python data webgl works on his machine now %time funky_func() %timeit funky_func() %%time %%timeit funky_func() %%prun -s cumulative -a 5 funky_func() %load_ext snakeviz %snakeviz funky_func() %load_ext line_profiler %lprun -f funky_func funky_func() %load_ext emory_profiler %memit funky_func() %memit? view -> cell toolbar -> edit metadata tag cells (named cells) scroll to tag in n seconds goto('bottom', time=10) goto('one', then-{'tags': 'two', 'then': {'tags': 'three}}) class GotoException(Exception): run cell magic (with javascript code) goto('x', 'y') can break loop across multiple cells # another cell it = range(10) val = next(it) # another cell loop_begin tag # another cell try: val = next(it) except Exception: goto('loopend') else: goto('loopbegin') # another cell loopend tag goto(tagname) $ jupyter lab next generation of jupyter notebook alpha new user interface for jupyter notebook allowing users to arrange multiple jupyter notebooks, text editors, terminals, output areas, etc. on a single page with multiple panels and tabs in one integrated application. tiled panes a pane can have multiple tabs various jupyter notebooks various python various terminal windows (bash) various text editor windows run code interactively outside of a notebook in a "Code Console" and connect one to a text file right click on a markdown file and "open with..." a live markdown viewer very handy!!! double click on csv files to view them as a nicely formatted table drag and drop notebook cells within a notebook or between notebooks multicursor support (apple) large rewrite flexibility and speed sns.kdeplot? sns.plotting_context inspector in another tab: cool! does not bog down when viewing large amount of data lazy loading huge table https://phosphorusjs.github.io/examples/datagrid/ all done over a single connection with ssh forwarding xonsh (pronounced konsh) editor -> key map -> vim (or default, emacs, or subline text) openwith -> markdown preview live rendering of markdown how about for rst? editor language support: only those beginning with a-c fit on screen can rebind just about anything --------------------------------------------------------------- dask: parallel python multiprocessing concurrent import dask import dask.array as da a = da.random.random((2000, 2000), chunks=1000) a.visualize() (a + a.T).vizualize() (a + (a.T + 1)).vizualize() dask.visualize( (a + (a.T + 1)).sum(axis=0), (a + (a.T + 1)).sum(axis=1), (a + (a.T + 1)).sum(), ) dask.compute( (a + (a.T + 1)).sum(axis=0), (a + (a.T + 1)).sum(axis=1), (a + (a.T + 1)).sum(), ) from dask.diagnostics import Profiler, ProgressBar ProgressBar().register() with Profiler() as prof: dask.compute( (a + (a.T + 1)).sum(axis=0), (a + (a.T + 1)).sum(axis=1), (a + (a.T + 1)).sum(), ) prof.visualize() bokeh is a plotting library bokeh is pronounced like bo-kay wp:Bokeh wp: prefix means Wikipedia To get good answers, consider following the advice in the links below. http://catb.org/~esr/faqs/smart-questions.html http://web.archive.org/web/20090627155454/www.greenend.org.uk/rjk/2000/06/14/quoting.html bokeh.pydata.org/en/latest/ from dask.distributed import Client client = Client() client compare to celery with rabbitmq 127.0.0.1:88787/profile dask stuff ref = dask.compute( (a + (a.T + 1)).sum(axis=0), (a + (a.T + 1)).sum(axis=1), (a + (a.T + 1)).sum() ) MacBook Pro (Retina, 15-inch, Mid 2014) 2.5 GHz intel core i7 16GB 1600MHz DDR3 nvidia geforce gt 750M 2048 MB intel iris pro 1536 MB from concurrent import futures ex = futures.ThreadPoolExecturo() import time def slow_inc(x): time.sleep(1) return x + 1 future = ex.submit(slow_inc, 1) future.result() results = ex.map(slow_inc, range(100)) # rvs = [x.result() for x in results] results = ex.map(slow_inc, range(100)) list(results) f = client.submit(slow_inc, 1) f.done() g = client.submit(slow_inc, f) g g.result() client.map python is growing https://stackoverflow.blog/2017/09/06/incredible-growth-python/ https://stackoverflow.blog/2017/09/14/python-growing-quickly/ python's scientific stack astropy boipython dipy nipy sunpy scikit learn statsmodel sympy networkx scikit-image pymc3 xarray bokeh matplotlib pandas scipy dask ipython numpy jupyter cython numba python python still limited by gil and harder to scale limited to a single thread limited to in-memory data dask designed to parallelize the python ecosystem handles complex algorithms co-developed with pandas/sklearn/juptyer teams familiar apis for python users scales scales from multicore to 1000-node clusters resilience, responsive, and real-time parallelizes numpy, pandas, sklearn satisfies subset of these apis uses these libraries internally co-developed with these teams task scheduler supports custom algorithms parallelize existing code build novel real-time systems arbitrary task graphs with data dependencies same scalability demo high level: scaling pandas same pandas look and feel uses pandas under the hood scales nicely onto many machines low level: arbitrary task scheduling parallelize normal python code build custom algorithms react real-time demo developed with dask-kubernetes google compute engine github.com/dask/dask-kubernetes Standard Dask Demo https://youtube.com/watch?v=ods97a5Pzw0 why do people choose dask familiar with python drop-in numpy/pandas/sklearn apis native memory environment easy debugging and diagnostics have complex problems parallelize existing code without extension rewrites sophisticated algorithms and systems real-time response to small-daa scales up and down scales to 1000-node clusters also runs cheaply on a laptop import pandas as pd import dask.dataframe as dd http://thefaradayproject.com/ continuum analytics changed their name to anaconda negative decimal (radix is -10) From betsy at python.org Thu Sep 28 17:06:07 2017 From: betsy at python.org (Betsy Waliszewski) Date: Thu, 28 Sep 2017 14:06:07 -0700 Subject: [CentralOH] PyCon 2018 CFP is Open! Message-ID: Hi, Woohoo! The PyCon 2018 CFP is open! Here is a link to all the information: https://us.pycon.org/2018/speaking/ - Tutorial proposals ? deadline is 24 November 2017 AoE. - Talk, Poster, and Education Summit proposals ? deadline is 3 January 2018 AoE. If you know of a company who might be interested in sponsoring, feel free to share our prospectus: https://us.pycon.org/2018/sponsors/prospectus/. An email intro to me would be wonderful. Cheers, Betsy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jep200404 at columbus.rr.com Fri Sep 29 17:23:36 2017 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Fri, 29 Sep 2017 17:23:36 -0400 Subject: [CentralOH] =?utf-8?q?2017-09-28_=E9=81=93=E5=A0=B4_Scribbles_?= =?utf-8?b?76SY5pu4L+aDoeaWhz8gcmVjdXJzaXZlIHByaW1lIHNlYXJjaDsgd2F0Y2hp?= =?utf-8?q?ng_refactoring=3B_coming_events?= Message-ID: <20170929172336.038a6643.jep200404@columbus.rr.com> https://github.com/joseph-albrecht/Project-Euler https://github.com/joseph-albrecht/Project-Euler/blob/master/Problem%203%20-%20Largest%20Prime%20Factor.ipynb doj at sbc:~$ cat .gitconfig ... [alias] co = checkout ci = commit st = status br = branch hist = log --pretty=format:\"%h %ad %s%d [%an]%n%P parents%n%T tree%n\" --graph --date=iso hist = log --pretty=format:\"%h %ad %s%d [%an]%n\" --graph --date=short hist = log --pretty=format:\"%h %ad %cd %s%d [%an]%n\" --graph --date=iso-local hist = log --pretty=format:\"%h %ad %s%d [%an]\" --graph --date=iso-local hist = log --pretty=format:\"%h %ad %s%d [%an]\" --graph --date=raw hist = log --pretty=format:\"%h %ad %cd %s%d [%an]\" --graph --date=format-local:'%Y-%m-%d %H:%M:%S' hist = log --pretty=format:\"%h %ad %s%d [%an]\" --graph --date=format-local:'%Y-%m-%d %H:%M:%S' type = cat-file -t dump = cat-file -p df = diff --color dw = diff --color --word-diff dm = difftool -t meld dy = difftool -t meld -y doj at sbc:~$ git clone https://github.com/james-prior/kraftur-math-game.git cd kraftur-math-game git co jep-dev git hist --color=always | grep --color=always '([^)].*)\|$' git hist | tac | awk 'NR > 1 {print $2}' | while read h; do git difftool -t meld -y "$h"^ "$h" simpleMathGame.py;done recent notebooks: (not all are posted yet) http://nbviewer.jupyter.org/github/james-prior/cohpy/blob/master/20170927-dojo-qtconsole-not-happy.ipynb http://nbviewer.jupyter.org/github/james-prior/cohpy/blob/master/20170928-dojo-iterating-over-dictionary-gets-keys.ipynb http://nbviewer.jupyter.org/github/james-prior/cohpy/blob/master/20170928-dojo-iter-sentinel-else.ipynb http://nbviewer.jupyter.org/github/james-prior/cohpy/blob/master/20170928-dojo-refactoring-kraftur-math-game.ipynb http://nbviewer.jupyter.org/github/james-prior/cohpy/blob/master/20170928-dojo-references-map-zip_longest-fstrings.ipynb wp:The Unix Programming Environment wp: prefix means Wikipedia To get good answers, consider following the advice in the links below. http://catb.org/~esr/faqs/smart-questions.html http://web.archive.org/web/20090627155454/www.greenend.org.uk/rjk/2000/06/14/quoting.html V. Anton Spraul's Think Like a Programmer, Python Edition http://www.linuxjournal.com/content/v-anton-sprauls-think-programmer-python-edition Open source-based business lessons from a seasoned CEO https://opensource.com/article/17/9/qa-evan-powell http://thefaradayproject.com/concept/ xdpyinfo | less Facebook Relents on React.js License Issue http://windowsitpro.com/open-source/facebook-relents-reactjs-license-issue autonomous car uses Python Donkey Car http://www.donkeycar.com/ Joint Meetup with VR Columbus: VR demo and robotics racing in the works https://www.meetup.com/AutonomyHub/events/243521906/ Ohio Linuxfest 2017 2017-09-29 to 2017-09-30 Columbus Code Camp 2017-10-14 http://columbuscodecamp.com/ DevOps Days https://www.devopsdays.org/events/2017-ohio/ Wednesday, Nov 8, 2017 - Thursday, Nov 9, 2017 The Bluestone Machine Vision with Python openmv (open machine vision) https://openmv.io/ https://github.com/openmv/openmv (opencv is too slow) What are the 7 stages of becoming a Python programmer? The 7 stages of becoming a Go programmer https://opensource.com/article/17/9/seven-stages-becoming-go-programmer Microsoft is Doing to ?Open Source? (OSI) What it Did to W3C http://techrights.org/2017/09/28/osi-and-microsoft/ https://www.reddit.com/r/dailyprogrammer/comments/6zvjre/20170913_challenge_331_intermediate_sum_of_digits/ From jep200404 at columbus.rr.com Fri Sep 29 18:21:53 2017 From: jep200404 at columbus.rr.com (jep200404 at columbus.rr.com) Date: Fri, 29 Sep 2017 18:21:53 -0400 Subject: [CentralOH] =?utf-8?q?2017-08-28_=E6=9C=83=E8=AD=B0_Scribbles_?= =?utf-8?b?76SY5pu4L+aDoeaWhz86IE5laWwgTHVkYmFuIFB5dGhvbiB2cyB0aGUgSGFy?= =?utf-8?q?dware=3B_freebsd_raspberry_pi2_cnc_real_I/O?= Message-ID: <20170929182153.0013ceed.jep200404@columbus.rr.com> Thanks to christoph baker and pillar for their generous hospitality there was pizza, salad, cookies, and beverages for all devopsdays ohio in columbus 2017-11-08 and 2017-11-09 https://www.devopsdays.org/events/2017-ohio/welcome/ 29 folks columbus state software development python database admin work for state of ohio powershell python interested in anyone who has used ironpython for integration with C#/.net bioinformatics at ohio state python is for 90% of what I do R is for other 10% september erik welch will be presenting tips and tricks for jupyter notebooks october len and mike handler python data analysis temperature and humidity sensor graph it in real time len did back end mike did front end dallas one-wire interface sensor first monday in december for social maybe meet at smokehouse brewing company january travis will talk about async await and curio library which was written by david beazley (dabeaz) ############################################################################### Python vs the Hardware by Neil Ludban https://bitbucket.com/nludban/python-vs-the-hardware/ private repo? theme: python and embedded systems several small python / freebsd / raspberry pi projects He used Raspberry Pi 2, because that version is best supported by FreeBSD. usb hid i2c gpio part 1 usb hid build cnc machine free usb game controller 12 buttons got it to work with python https://pypi.python.org/pypi/libusb1 adafruit circuitpython https://circuitpython.readthedocs.io beta micropython derivative attaches as USB device keyboard mouse midi feather m0 express metro m0 express the "right way" == the "wrong way" supposed to - get a device driver - find existing - modify it - write from scratch - configure kernel - one of - write python module to interface to kernel device - configure x11 to ignore device or - configure x11 to enable device - write python module to interface to x11 device - application now revolves around x11 1-bit message for a button click sudo usbhidctl -f /dev/uhid1 -a -v -l import usb1 import libusb1 part 1 freebsd.org 11.1 july 2017 - production servers - high end embedded systems products based on freebsd - mac os x - juniper routers - netflix - game consoles: ps3, ps4, switch freebsd raspbian all i/o works on RPi2, dunno about RPi3 x11 not running yet ARM6/7 ports confusion no official (RPi) pre-built packages native builds are slow #hardware build tools usb data capture www.saleae.com mixed signal oscilloscope www.rigolna.com agilent 54622D mixed signal oscilloscope - circa 2000 - 2 analog - 16 digital - some serial protocol support - good parallel bus support - excellent real-time hi-res display - https://www.youtube.com/watch?v=aVNfFewFn_Y #part 3 - i2c for python on freebsd found at microcenter pimoroni four-letter-phat $12 four 15-segment LED displays f007347 pimoroni - pure python driver from the vendor https://github.com/pimoroni/fourletter-phat - uses https://github.com/pimoroni/py-smbus - which uses linux #include freebsd - wp:SMB - SMB is a subset of I2C - no python i2c or smbus for freebsd - freebsd i2c is spelled iic man 4 iic only did single byte reads and writes did not need to do multi-byte reads or writes plan #1 - python module iic.py - class IIC implemented with - file descriptor from os.open - I/O using os.read, os.write - C extension module (_bsdiic.c) for ioctl - I2CSADDR - dummy module (smbus.py) - class SMBus with methods: - open() - set_address() - write_i2c_block_data() _bsdiic.c #include #includ static PyObject * foo( PyObject *self, PyObject *args ) { PyArg_ParseTuple(args, "ii", &fd, &addr); PyErr_SetFromErrno(PyExc_IOError); Py_INCREF(Py_None); return Py_None; } static PyMethodDef foo_methods[] = { { "set_slave_address", foo, METH_VARARGS, "I2CSADDR" }, { 0 } }; static struct PyModuleDef foo_module = { PyModuleDef_HEAD_INIT, "_bsdiic", NULL, -1, foo_methods, 0 }; PyObject * PyInit__bsdiic(void) { PyObject *m = PyModule_Create(&bsdiic_module); } smbus.py from iic ipmort IIC class SMBus: setup.py #!/usr/bin/env python3.6 from distutils.core import setup, Extension /usr/src/sys/arm/broadcom/bcm2835/bcm2835_bsc.c NOTE: Important detail uncovered by web searches: this kernel driver does not implement read and write. static device_method_t bcm_bsc_methods[] = { ... /* iicbus interface */ DEVMETHOD(iicbus_reset, bcm_bsc_iicbus_reset), DEVMETHOD(iicbus_callback, iicbus_null_callback), DEVMETHOD(iicbus_transfer, bcm_bsc_transfer), ... (what does my C code read talking to thermal sensor use?) Plan #2 - Python class IIC - Verify data is bytes - remember slave_address - C extension adds read and write functions using I2CRDWR ioctl iic.py _bsdiic.c static PyObject * bsdiic_write( PyObject *self, PyObject *args ) { PyObject *obj; !PyArg_ParseTuple(args, "ii0", &fd, &addr, &obj)) PyBytes_Check(obj) PyBytes_Size(obj) PyErr_SetString(PyExc_ValueError, "hello world"); PyBytes_Size(obj); PyBytes_AsString(obj); PyErr_SetFromErrno(PyExc_OSError); Py_INCREF(Py_None); } _bsdiic.c had to shift I2C address one bit left PyBytes(obj) (uint8_t *)PyBytes_AsString(Obj) demo has pirate theme https://github.com/pimoroni/fourletter-phat/tree/master/examples https://github.com/pimoroni/fourletter-phat/blob/master/examples/demo.py https://github.com/pimoroni/fourletter-phat/blob/master/examples/countdown.py # Part 4 GPIO for Python on FreeBSD - found pimoroni displayotron hat at microcenter for about $25 https://shop.pimoroni.com/products/display-o-tron-hat https://github.com/pimoroni/displayotron - lcd driver (spi data; gpio select and reset) - capacitive touch interface (i2c data; gpio ???) - backlight driver (i2c data) - already have i2c - need gpio - can implement (slow) SPI over GPIO existing python packages - https://pypi.python.org/pypi/RPi.GPIO - Raspberry Pi and Linux specific - memory mapped I/O device access - mostly C code (~3k lines) - https://github.com/gonzoua/freebsd-gpio - not maintained, suggests following: - https://github.com/evadot/fbsd_gpio_py - wrapper for libgpio - uses CFFI (C foreign Function Interface for Python) https://cffi.readthedocs.io/ looks like write by someone trying out a bunch of new things in Python without catching on to what one is really supposed to do as opposed to what one can do neil hopes he never sees CFFI again - throws exceptions while handling exceptions - clunky interface pin = getattr(controller, 'pin 42') (arduinoesque) - properties with major side effects: pin.name = 'foobar' freebsd options - search for detected devices - kernel supplied devices - system header files dmesg | grep -i gpio ls -l /dev/gpio* /dev/gpioc0 man 8 gpioctl ;# note wrong section sudo gpioctl -l -v man 3 gpio came out of netbsd originally? more questions - sysctl or libgpio? - sysctl is low level - libgpio requires gpio_handle_t - result of setting conflicting flags? - last takes precedence? - error? - after more searching: - nothing gained with libgpio - very thin wrapper over ioctl()s - kernel device driver has implicit priorities additional cpio knobs supported through sysctl(8) /usr/src/lib/libgpio/gpio.c /usr/src/sys/arm/broadcom/bcm2835/bcm2835_gpio.c sysctl -a | grep -i gpio came from intel so he wrote is own interface a gpio example NOTE: The biggest requirement is a clean Pythonic API. Test first by writing a small but complete application. (Everything should be built top-down, except the first time.) ("Facade" design pattern.) import gpio c = gpio.open('/dev/gpioc0') for pin in c.list_pins(): print(pin) bcm2 = c.get_pin(number=2) # (i2c SDA / header pin 3) bcm3 - c.get_pin(name='pin 3') # (i2c SCL / header pin 5) # bug in following? # should write initial value before configuring for output? with bcm2.configure() as pin_config: pin_config.output = True pin_config.initial = 0 # Why not 1? with bcm3.configure() as pin_config: pin_config.output = True pin_config.initial = 0 # Why not 1? while True: bcm2.set(1) # 1,0 bcm3.set(1) # 1,1 bcm2.set(0) # 1,0 bcm3.set(0) # 0,0 # above loop ran at 10.7 kHz (four pin.set() calls) # neil did not notice jitter but first, some refactoring NOTE: Created a top-level "pybsd" package to contain iic and gpio sub-projects. To avoid changing application code smbus and RPi stay at the top level pybsd/gpio/__init__.py NOTE: The package __init__ exports the public interfaces. pybsd/gpio/constants.py NOTE: Copying constant definitions from C to a more convenient namespace. XXX import _bsdgpio.constants as ? pybsd/gpio/controller.py NOTE: The GPIO Controller manages the file descriptor and provides access to individual pins. pybsd/gpio/pin.py NOTE: The Pin class is supported by Flags, Capabilities, and Configuration. NOTES: Configuration extends the read-only Flags properties with setter properties. class Flags @property class Capabilities(Flags) class Configuration(Flags) @Flags.input.setter def input(self, yes_no) class Pin _bsdgpio.c NOTE: Mostly boilerplate; abridged to highlight new techniques. #include #include static PyObject * bsdgpio_get_pin_config( PyObject *self, PyObject *args ) { PyArg_ParseTuple(args, "ii", &fd, &pin_number) PyErr_SetFromErrno(PyExc_IOError) PyObject *t = PyTuple_New(3); PyTuple_SetItem(t, 0, PyBytes_FromString(gp.gp_name)); PyTuple_SetItem(t, 1, PyLong_FromLong(gp.gp_caps)); PyTuple_SetItem(t, 2, PyLong_FromLong(gp.gp_flags)); } PyObject *rv = (gr.gp_value ? Py_True : Py_False); Py_INCREF(rv); return rv; PyObject * PyInit__bsdgpio(void) { PyObject *m = PyModule_Create(&bsdgpio_module); PyModule_AddIntConstant(m, "PIN_INPUT", GPIO_PIN_INPUT); PyModule_AddIntConstant(m, "PIN_OUTPUT", GPIO_PIN_OUTPUT); PyModule_AddIntConstant(m, "PIN_OPEN_DRAIN", GPIO_PIN_OPEN_DRAIN); PyModule_AddIntConstant(m, "PIN_PUSHPULL", GPIO_PIN_PUSHPULL); PyModule_AddIntConstant(m, "PIN_TRISTATE", GPIO_PIN_TRISTATE); }