From nick.bollweg at gmail.com Mon Jun 1 08:24:49 2015 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Mon, 01 Jun 2015 12:24:49 +0000 Subject: [IPython-dev] Is there a way to open a notebook via one click in tmpnb.org? In-Reply-To: References: Message-ID: I see a lot of problems with this, but would desperately love to see it happen. If some route on tmpnb used the same url scheme as nbviewer, and we knew which kernels were there, it would be possible to make every notebook up there viewable... Maybe even use nbviewer as a first pass filter. The issues: - no real knowledge of which kernels are available in tmpnb (working on this) - no concise metadata for what should be available when the kernel starts (requirements.txt) - many notebooks use other things (code, assets) stored along with them... What should we being along? For repos, that would be "easy," for raw URLs that would be "impossible". - At the end of a session... So what? Likely will never add authentication to tmpnb, so it would be hard to get your data back out... Fine for playground stuff, but... On 08:11, Mon, Jun 1, 2015 Jason Moore wrote: > I'd like to provide a url in a lecture that the audience can enter in > their browser opening tmpnb.org with a particular notebook (i.e. without > downloading the notebook, uploading, and then opening it). Is that possible? > > Jason > moorepants.info > +01 530-601-9791 > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cyrille.rossant at gmail.com Mon Jun 1 08:41:12 2015 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Mon, 1 Jun 2015 14:41:12 +0200 Subject: [IPython-dev] Is there a way to open a notebook via one click in tmpnb.org? In-Reply-To: References: Message-ID: Perhaps another option could be to use sharecell and thebe? https://github.com/rgbkrk/share-cell https://github.com/oreillymedia/thebe I imagine one could create a pure client-side JS app that downloads a public notebook and displays one interactive cell per notebook cell. 2015-06-01 14:24 GMT+02:00 Nicholas Bollweg : > I see a lot of problems with this, but would desperately love to see it > happen. > > If some route on tmpnb used the same url scheme as nbviewer, and we knew > which kernels were there, it would be possible to make every notebook up > there viewable... Maybe even use nbviewer as a first pass filter. > > The issues: > - no real knowledge of which kernels are available in tmpnb (working on > this) > - no concise metadata for what should be available when the kernel starts > (requirements.txt) > - many notebooks use other things (code, assets) stored along with them... > What should we being along? For repos, that would be "easy," for raw URLs > that would be "impossible". > - At the end of a session... So what? Likely will never add authentication > to tmpnb, so it would be hard to get your data back out... Fine for > playground stuff, but... > > > On 08:11, Mon, Jun 1, 2015 Jason Moore wrote: >> >> I'd like to provide a url in a lecture that the audience can enter in >> their browser opening tmpnb.org with a particular notebook (i.e. without >> downloading the notebook, uploading, and then opening it). Is that possible? >> >> Jason >> moorepants.info >> +01 530-601-9791 >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From rgbkrk at gmail.com Mon Jun 1 10:05:50 2015 From: rgbkrk at gmail.com (Kyle Kelley) Date: Mon, 1 Jun 2015 10:05:50 -0400 Subject: [IPython-dev] Is there a way to open a notebook via one click in tmpnb.org? In-Reply-To: References: Message-ID: > I see a lot of problems with this I assume you're saying the inherent cross site scripting issue with this. If we ever enabled it, it wouldn't be a default on tmpnb (the code, not the service). The related issue for this is https://github.com/jupyter/docker-demo-images/issues/16 (I should open a new one...) If someone wants to add it on to jupyter/tmpnb, it's a matter of making the redirect uri an optional GET param for the spawn handler (that should be disabled at the server level by default): https://github.com/jupyter/tmpnb/blob/d74561c7ff80721b4aefea66ab435459e59dfe9a/orchestrate.py#L68 > Perhaps another option could be to use sharecell and thebe :) I like how if I leak something on Twitter it eventually reaches the mailing list. Share cell was built for a demo this weekend, but (expectedly so) people are already using for silly little snippets: https://goo.gl/02ekaZ Pairing it up with github.js is quite neat, I did a bit of a mockup with it earlier but was unsure about a reasonable standard of handling gists. Mimicking nbviewer might be ok. The nice thing there is that it can all be done client side. ;) The difference between a link to a sharecell "page" and a notebook is that javascript output on sharecell doesn't show until you run the snippet whereas the notebook server will display any trusted notebook. -- Kyle On Mon, Jun 1, 2015 at 8:41 AM, Cyrille Rossant wrote: > Perhaps another option could be to use sharecell and thebe? > https://github.com/rgbkrk/share-cell > https://github.com/oreillymedia/thebe > > I imagine one could create a pure client-side JS app that downloads a > public notebook and displays one interactive cell per notebook cell. > > 2015-06-01 14:24 GMT+02:00 Nicholas Bollweg : > > I see a lot of problems with this, but would desperately love to see it > > happen. > > > > If some route on tmpnb used the same url scheme as nbviewer, and we knew > > which kernels were there, it would be possible to make every notebook up > > there viewable... Maybe even use nbviewer as a first pass filter. > > > > The issues: > > - no real knowledge of which kernels are available in tmpnb (working on > > this) > > - no concise metadata for what should be available when the kernel starts > > (requirements.txt) > > - many notebooks use other things (code, assets) stored along with > them... > > What should we being along? For repos, that would be "easy," for raw URLs > > that would be "impossible". > > - At the end of a session... So what? Likely will never add > authentication > > to tmpnb, so it would be hard to get your data back out... Fine for > > playground stuff, but... > > > > > > On 08:11, Mon, Jun 1, 2015 Jason Moore wrote: > >> > >> I'd like to provide a url in a lecture that the audience can enter in > >> their browser opening tmpnb.org with a particular notebook (i.e. > without > >> downloading the notebook, uploading, and then opening it). Is that > possible? > >> > >> Jason > >> moorepants.info > >> +01 530-601-9791 > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Kyle Kelley (@rgbkrk ; lambdaops.com, developer.rackspace.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Jun 1 10:36:18 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Mon, 1 Jun 2015 07:36:18 -0700 Subject: [IPython-dev] IPython 3.1.0 headings pop-up In-Reply-To: References: Message-ID: <347AB73D-63A3-4560-9DAE-E391A4A9DF5F@gmail.com> > On May 31, 2015, at 11:17, Jose Gomez-Dans wrote: > > Hi, > > In the latest IPython version, there's a pop up that jumps out whenever you select a cell and indicate that it will be a heading, and telling you to indicate the hierarchy of the heading using markdown syntax. Now, after a while this gets quite annoying. The pop-up is annoying on purpose so that user loose the habit of selecting ?Heading? and click ?Markdown?. This behave as expected, thanks for confirming. > Is there some way to turn this off? Also, shouldn't it turn itself off by default after the first N times? > > IPython no longer uses special heading cells. Instead, **write your headings in Markdown cells** using # characters: Are you by any chance still selecting Heading ? If so click Markdown. Maybe we should make the text use a blink tag and play an annoying sound if people still click on Heading ? Or be clearer on the text. If you are still clicking Heading by 5.0 don?t worry the option will probably be removed. ? M Keep Calm and Click Markdown. > Thanks! > Jose > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Jun 1 10:41:41 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Mon, 1 Jun 2015 07:41:41 -0700 Subject: [IPython-dev] How to display markdown text in DataFrame in Notebook In-Reply-To: References: <88384F20-027A-4C24-BC94-4EA106EE59B9@gmail.com> Message-ID: > Hi Matthias, > > On Sat, May 30, 2015 at 6:48 PM, Matthias Bussonnier > wrote: > > For some reason people don?t try, or don?t read the doc: > > Thanks for the explanation, and my apologies for the incorrect reply. I did read the docs and asked on gitter, only I didn't recheck since v2, and _repr_markdown_ as well as IPython.display.Markdown were introduced in v3. No problem, we are bad at writing docs, I try to avoid gitter,I?m bad in chat room, and I find it difficult to dig into for archive. > > > > Which is the bad solution also, register a display_formatter for a specific object type: > > https://nbviewer.jupyter.org/github/ipython/ipython-in-depth/blob/bigsplit/examples/IPython%20Kernel/Custom%20Display%20Logic.ipynb#Adding-IPython-display-support-to-existing-objects > > > Thanks for sharing the ipython-in-depth link. May I suggest to also refer to it from http://ipython.org/documentation.html , it seems to have useful explanations that are not contained in the rest of the docs (at least as far as I could tell). Well it would be better to just improve our docs, the tutorial we get out of sync automatically. We?ll try to improve that. ? M > > Anton > > ? > M > > >> >> Best, >> Anton >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > ? > M > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgbkrk at gmail.com Mon Jun 1 12:06:42 2015 From: rgbkrk at gmail.com (Kyle Kelley) Date: Mon, 1 Jun 2015 12:06:42 -0400 Subject: [IPython-dev] Is there a way to open a notebook via one click in tmpnb.org? In-Reply-To: References: Message-ID: Whoops, just realized I misunderstood your initial question Jason. Sorry! There's not a way to bring in content from GitHub (or anywhere over the network) from the hosted tmpnb.org because of network lock down. You can pass anything you want in through JavaScript and file upload (as you're doing by bringing the notebook in). This means that if there was any integration between client side tooling, e.g. github.js ( https://github.com/michael/github), we could display all manner of things from GitHub within the notebook environment. There's a strong demand for this, including myself. The starter for an endeavor like this (to do it the right way), would be to write a client side contents service like jupyter-drive ( https://github.com/jupyter/jupyter-drive) whose backend was github. prose.io already has a nice setup for simple editing using github.js and it's surprisingly performant. The missing piece, to me, is data and scripts being available. Considering the notebooks and kernels behave with normal file system operations, it wouldn't have knowledge of the rest of the repository. Stuff to chew on. -- Kyle On Mon, Jun 1, 2015 at 10:05 AM, Kyle Kelley wrote: > > I see a lot of problems with this > > I assume you're saying the inherent cross site scripting issue with this. > If we ever enabled it, it wouldn't be a default on tmpnb (the code, not the > service). The related issue for this is > https://github.com/jupyter/docker-demo-images/issues/16 (I should open a > new one...) > > If someone wants to add it on to jupyter/tmpnb, it's a matter of making > the redirect uri an optional GET param for the spawn handler (that should > be disabled at the server level by default): > https://github.com/jupyter/tmpnb/blob/d74561c7ff80721b4aefea66ab435459e59dfe9a/orchestrate.py#L68 > > > Perhaps another option could be to use sharecell and thebe > > :) I like how if I leak something on Twitter it eventually reaches the > mailing list. Share cell was built for a demo this weekend, but > (expectedly so) people are already using for silly little snippets: > https://goo.gl/02ekaZ > > Pairing it up with github.js is quite neat, I did a bit of a mockup with > it earlier but was unsure about a reasonable standard of handling gists. > Mimicking nbviewer might be ok. The nice thing there is that it can all be > done client side. ;) > > The difference between a link to a sharecell "page" and a notebook is that > javascript output on sharecell doesn't show until you run the snippet > whereas the notebook server will display any trusted notebook. > > -- Kyle > > On Mon, Jun 1, 2015 at 8:41 AM, Cyrille Rossant > wrote: > >> Perhaps another option could be to use sharecell and thebe? >> https://github.com/rgbkrk/share-cell >> https://github.com/oreillymedia/thebe >> >> I imagine one could create a pure client-side JS app that downloads a >> public notebook and displays one interactive cell per notebook cell. >> >> 2015-06-01 14:24 GMT+02:00 Nicholas Bollweg : >> > I see a lot of problems with this, but would desperately love to see it >> > happen. >> > >> > If some route on tmpnb used the same url scheme as nbviewer, and we knew >> > which kernels were there, it would be possible to make every notebook up >> > there viewable... Maybe even use nbviewer as a first pass filter. >> > >> > The issues: >> > - no real knowledge of which kernels are available in tmpnb (working on >> > this) >> > - no concise metadata for what should be available when the kernel >> starts >> > (requirements.txt) >> > - many notebooks use other things (code, assets) stored along with >> them... >> > What should we being along? For repos, that would be "easy," for raw >> URLs >> > that would be "impossible". >> > - At the end of a session... So what? Likely will never add >> authentication >> > to tmpnb, so it would be hard to get your data back out... Fine for >> > playground stuff, but... >> > >> > >> > On 08:11, Mon, Jun 1, 2015 Jason Moore wrote: >> >> >> >> I'd like to provide a url in a lecture that the audience can enter in >> >> their browser opening tmpnb.org with a particular notebook (i.e. >> without >> >> downloading the notebook, uploading, and then opening it). Is that >> possible? >> >> >> >> Jason >> >> moorepants.info >> >> +01 530-601-9791 >> >> _______________________________________________ >> >> IPython-dev mailing list >> >> IPython-dev at scipy.org >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Kyle Kelley (@rgbkrk ; lambdaops.com, > developer.rackspace.com) > -- Kyle Kelley (@rgbkrk ; lambdaops.com, developer.rackspace.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From moorepants at gmail.com Mon Jun 1 12:32:50 2015 From: moorepants at gmail.com (Jason Moore) Date: Mon, 1 Jun 2015 09:32:50 -0700 Subject: [IPython-dev] Is there a way to open a notebook via one click in tmpnb.org? In-Reply-To: References: Message-ID: Thanks Kyle, The issue: https://github.com/jupyter/docker-demo-images/issues/16 is exactly what I was hoping existed. It'd be nice to simply click a button on a notebook in nbviewer and it opens up in a tmpnb.org instance. But obviously from the conversation there is more than meets the eye. Jason moorepants.info +01 530-601-9791 On Mon, Jun 1, 2015 at 9:06 AM, Kyle Kelley wrote: > Whoops, just realized I misunderstood your initial question Jason. Sorry! > > There's not a way to bring in content from GitHub (or anywhere over the > network) from the hosted tmpnb.org because of network lock down. You can > pass anything you want in through JavaScript and file upload (as you're > doing by bringing the notebook in). This means that if there was any > integration between client side tooling, e.g. github.js ( > https://github.com/michael/github), we could display all manner of things > from GitHub within the notebook environment. There's a strong demand for > this, including myself. > > The starter for an endeavor like this (to do it the right way), would be > to write a client side contents service like jupyter-drive ( > https://github.com/jupyter/jupyter-drive) whose backend was github. > prose.io already has a nice setup for simple editing using github.js and > it's surprisingly performant. > > The missing piece, to me, is data and scripts being available. Considering > the notebooks and kernels behave with normal file system operations, it > wouldn't have knowledge of the rest of the repository. > > Stuff to chew on. > > -- Kyle > > > > > On Mon, Jun 1, 2015 at 10:05 AM, Kyle Kelley wrote: > >> > I see a lot of problems with this >> >> I assume you're saying the inherent cross site scripting issue with this. >> If we ever enabled it, it wouldn't be a default on tmpnb (the code, not the >> service). The related issue for this is >> https://github.com/jupyter/docker-demo-images/issues/16 (I should open a >> new one...) >> >> If someone wants to add it on to jupyter/tmpnb, it's a matter of making >> the redirect uri an optional GET param for the spawn handler (that should >> be disabled at the server level by default): >> https://github.com/jupyter/tmpnb/blob/d74561c7ff80721b4aefea66ab435459e59dfe9a/orchestrate.py#L68 >> >> > Perhaps another option could be to use sharecell and thebe >> >> :) I like how if I leak something on Twitter it eventually reaches the >> mailing list. Share cell was built for a demo this weekend, but >> (expectedly so) people are already using for silly little snippets: >> https://goo.gl/02ekaZ >> >> Pairing it up with github.js is quite neat, I did a bit of a mockup with >> it earlier but was unsure about a reasonable standard of handling gists. >> Mimicking nbviewer might be ok. The nice thing there is that it can all be >> done client side. ;) >> >> The difference between a link to a sharecell "page" and a notebook is >> that javascript output on sharecell doesn't show until you run the snippet >> whereas the notebook server will display any trusted notebook. >> >> -- Kyle >> >> On Mon, Jun 1, 2015 at 8:41 AM, Cyrille Rossant < >> cyrille.rossant at gmail.com> wrote: >> >>> Perhaps another option could be to use sharecell and thebe? >>> https://github.com/rgbkrk/share-cell >>> https://github.com/oreillymedia/thebe >>> >>> I imagine one could create a pure client-side JS app that downloads a >>> public notebook and displays one interactive cell per notebook cell. >>> >>> 2015-06-01 14:24 GMT+02:00 Nicholas Bollweg : >>> > I see a lot of problems with this, but would desperately love to see it >>> > happen. >>> > >>> > If some route on tmpnb used the same url scheme as nbviewer, and we >>> knew >>> > which kernels were there, it would be possible to make every notebook >>> up >>> > there viewable... Maybe even use nbviewer as a first pass filter. >>> > >>> > The issues: >>> > - no real knowledge of which kernels are available in tmpnb (working on >>> > this) >>> > - no concise metadata for what should be available when the kernel >>> starts >>> > (requirements.txt) >>> > - many notebooks use other things (code, assets) stored along with >>> them... >>> > What should we being along? For repos, that would be "easy," for raw >>> URLs >>> > that would be "impossible". >>> > - At the end of a session... So what? Likely will never add >>> authentication >>> > to tmpnb, so it would be hard to get your data back out... Fine for >>> > playground stuff, but... >>> > >>> > >>> > On 08:11, Mon, Jun 1, 2015 Jason Moore wrote: >>> >> >>> >> I'd like to provide a url in a lecture that the audience can enter in >>> >> their browser opening tmpnb.org with a particular notebook (i.e. >>> without >>> >> downloading the notebook, uploading, and then opening it). Is that >>> possible? >>> >> >>> >> Jason >>> >> moorepants.info >>> >> +01 530-601-9791 >>> >> _______________________________________________ >>> >> IPython-dev mailing list >>> >> IPython-dev at scipy.org >>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> > >>> > >>> > _______________________________________________ >>> > IPython-dev mailing list >>> > IPython-dev at scipy.org >>> > http://mail.scipy.org/mailman/listinfo/ipython-dev >>> > >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> >> >> >> -- >> Kyle Kelley (@rgbkrk ; lambdaops.com, >> developer.rackspace.com) >> > > > > -- > Kyle Kelley (@rgbkrk ; lambdaops.com, > developer.rackspace.com) > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Jun 1 13:22:50 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 1 Jun 2015 10:22:50 -0700 Subject: [IPython-dev] IPython 3.1.0 headings pop-up In-Reply-To: <347AB73D-63A3-4560-9DAE-E391A4A9DF5F@gmail.com> References: <347AB73D-63A3-4560-9DAE-E391A4A9DF5F@gmail.com> Message-ID: I believe that the keyboard shortcuts to create headings (1-6) still work without bringing up a dialog. On 1 June 2015 at 07:36, Matthias Bussonnier wrote: > > On May 31, 2015, at 11:17, Jose Gomez-Dans wrote: > > Hi, > > In the latest IPython version, there's a pop up that jumps out whenever > you select a cell and indicate that it will be a heading, and telling you > to indicate the hierarchy of the heading using markdown syntax. Now, after > a while this gets quite annoying. > > > The pop-up is annoying on purpose so that user loose the habit of > selecting ?Heading? and click ?Markdown?. > This behave as expected, thanks for confirming. > > Is there some way to turn this off? Also, shouldn't it turn itself off by > default after the first N times? > > > > IPython no longer uses special heading cells. Instead, **write your > headings in Markdown cells** using # characters: > > Are you by any chance still selecting Heading ? > If so click Markdown. > Maybe we should make the text use a blink tag > and play an annoying sound if people still click on Heading ? Or be > clearer on the text. > > If you are still clicking Heading by 5.0 don?t worry the option will > probably be removed. > ? > M > Keep Calm and Click Markdown. > > > > Thanks! > Jose > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Jun 1 13:25:52 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Mon, 1 Jun 2015 10:25:52 -0700 Subject: [IPython-dev] How to display markdown text in DataFrame in Notebook In-Reply-To: References: <88384F20-027A-4C24-BC94-4EA106EE59B9@gmail.com> Message-ID: Hi, > On May 30, 2015, at 13:00, Dong Ta wrote: > > Hi Matthias, > > After reading your answer and the notebook, I still can't figure out how to properly display markdown columns in a DataFrame. You have 2 well, maybe 3 solutions. the first 2 require you to write a function that create a markdown repr for panda data frame. ie, a function that will return a string like [1] First Header | Second Header ------------- | ------------- Content Cell | Content Cell Content Cell | Content Cell either you provide a path to panda so that data frames expose that (solution1) Or you register it as a markdown display formatter, (solution 2) The third is actually convert the markdown to html yourself, and set the content of the cell you like to the generated HTML. ? M [1]: roughly that: from IPython.display import Markdown df = pandas.DataFrame([['a','b','c'],['d','e','f']]) def df_row(row): return '|'.join(row[1]) def dfm(df): return Markdown('\n'.join(( '|'.join(map(str, df.columns)), '|'.join(map(lambda x: '-', df.columns)), '\n'.join(map(df_row, df.iterrows())) ))) dfm(df) > > Can you be more specific? > > Thanks a lot. > -DT > > On Sat, May 30, 2015 at 9:48 AM, Matthias Bussonnier > wrote: > >> On May 30, 2015, at 03:59, Anton Akhmerov > wrote: >> >> Somewhat paradoxically, there's no easy way to display markdown >> programmatically in ipython. You can manually create HTML out of >> markdown by using markdown2html filters from >> IPython.nbconvert.filters. > > > > For some reason people don?t try, or don?t read the doc: > > class o: > def _repr_markdown_(self): > return '[try it](https://try.jupyter.org )' > > o() > > > And we support arbitrary mime type, the _repr_*_ that exist are here just for convenience. > >> If you want a more systematic solution, I believe you could subclass >> the DataFrame > > Which is the bad solution also, register a display_formatter for a specific object type: > > https://nbviewer.jupyter.org/github/ipython/ipython-in-depth/blob/bigsplit/examples/IPython%20Kernel/Custom%20Display%20Logic.ipynb#Adding-IPython-display-support-to-existing-objects > > ? > M > > >> >> Best, >> Anton >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > ? > M > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From jgomezdans at gmail.com Thu Jun 4 07:56:46 2015 From: jgomezdans at gmail.com (Jose Gomez-Dans) Date: Thu, 4 Jun 2015 12:56:46 +0100 Subject: [IPython-dev] IPython 3.1.0 headings pop-up In-Reply-To: References: <347AB73D-63A3-4560-9DAE-E391A4A9DF5F@gmail.com> Message-ID: OK, OK.... I hung my head in shame... On 1 June 2015 at 18:22, Thomas Kluyver wrote: > I believe that the keyboard shortcuts to create headings (1-6) still work > without bringing up a dialog. > > On 1 June 2015 at 07:36, Matthias Bussonnier > wrote: > >> >> On May 31, 2015, at 11:17, Jose Gomez-Dans wrote: >> >> Hi, >> >> In the latest IPython version, there's a pop up that jumps out whenever >> you select a cell and indicate that it will be a heading, and telling you >> to indicate the hierarchy of the heading using markdown syntax. Now, after >> a while this gets quite annoying. >> >> >> The pop-up is annoying on purpose so that user loose the habit of >> selecting ?Heading? and click ?Markdown?. >> This behave as expected, thanks for confirming. >> >> Is there some way to turn this off? Also, shouldn't it turn itself off by >> default after the first N times? >> >> >> > IPython no longer uses special heading cells. Instead, **write your >> headings in Markdown cells** using # characters: >> >> Are you by any chance still selecting Heading ? >> If so click Markdown. >> Maybe we should make the text use a blink tag >> and play an annoying sound if people still click on Heading ? Or be >> clearer on the text. >> >> If you are still clicking Heading by 5.0 don?t worry the option will >> probably be removed. >> ? >> M >> Keep Calm and Click Markdown. >> >> >> >> Thanks! >> Jose >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paad.ruslan.korniichuk at gmail.com Thu Jun 4 11:04:39 2015 From: paad.ruslan.korniichuk at gmail.com (Ruslan Korniichuk) Date: Thu, 4 Jun 2015 17:04:39 +0200 Subject: [IPython-dev] KernelRestarter infinite loop Message-ID: Hi all, We are working on writing the *rk* administration utility (home page: https://github.com/korniichuk/rk). If the *rkscript* runs LESS than 3 seconds and kill itself (sys.exit(1)),- *jupyter notebook *working GOOD: KernelRestarter: restarting kernel (*1/5*) ... KernelRestarter: restarting kernel (*4/5*) But, if the *rkscript* runs LONGER than 3 seconds and kill itself (sys.exit(1)),- jupyter notebook working BAD (infinite loop): KernelRestarter: restarting kernel (*1/5*) KernelRestarter: restarting kernel (*1/5*) ... Variable *time_to_dead* = 3 sec (kernel heartbeat interval in seconds) in the *restarter.py* file. *How we can change heartbeat interval or do something to GOOD working of sys.exit(1), after the third second?* It is important for us, because we need to check a remote machine availability (ping), to check SSH login w/o password, to responsive to the excepts. --- Best regards, Ruslan Korniichuk -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Fri Jun 5 19:25:08 2015 From: benjaminrk at gmail.com (MinRK) Date: Fri, 5 Jun 2015 16:25:08 -0700 Subject: [IPython-dev] [ATTN: Kernel Authors] libzmq change breaks kernel heartbeat Message-ID: ATTN: Kernel authors, libzmq just released 4.0.6 and 4.1.1. Both of these include a change in zmq_proxy that breaks the kernel-side of our heartbeat machinery. Specifically, zmq_proxy cannot be called with a single REP socket for both ends. Currently, the message spec docs recommend the kernel-side of the heartbeat mechanism be implemented as zmq_device(FORWARDER, REP, REP). zmq_device has wrapped zmq_proxy since libzmq 3, in which case the first argument is actually ignored. This can be fixed by making the REP socket a ROUTER, which is what it should be anyway. There?s one caveat for this change, though: On very old systems with libzmq-2, ROUTERs don?t work in a FORWARDER device, only QUEUE. This is fine, and doesn?t really make a difference. To be fully compatible with libzmq-2.2 through 4.1, the heartbeat call should change from: zmq_device(FORWARDER, REP, REP) to: zmq_device(QUEUE, ROUTER, ROUTER) If you only need to support libzmq ? 3 (recommended), you can use: zmq_proxy(ROUTER, ROUTER) If you were using your own echo loop for the heartbeat socket instead of zmq_device, you shouldn?t be affected. This is really an implementation detail, not a protocol change, because it doesn?t affect how messages are sent or received by the kernel, only the internal event handling (REP is a restricted subclass of ROUTER, indistinguishable to peers). -MinRK ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Jun 8 19:45:42 2015 From: benjaminrk at gmail.com (MinRK) Date: Mon, 8 Jun 2015 16:45:42 -0700 Subject: [IPython-dev] [ATTN: Kernel Authors] libzmq change breaks kernel heartbeat In-Reply-To: References: Message-ID: Patches have been accepted to various zeromq branches, so this problem should not affect libzmq-4.0.7 or libzmq-4.1.2, once they are released. I would still recommend kernel authors make the changes proposed above, even though they will not be necessary after the next patch release of libzmq. -MinRK On Fri, Jun 5, 2015 at 4:25 PM, MinRK wrote: > ATTN: Kernel authors, > > libzmq just released 4.0.6 and 4.1.1. Both of these include a change in > zmq_proxy that breaks the kernel-side of our heartbeat machinery. > Specifically, zmq_proxy cannot be called with a single REP socket for > both ends. > > Currently, the message spec docs recommend the kernel-side of the > heartbeat mechanism be implemented as zmq_device(FORWARDER, REP, REP). > zmq_device has wrapped zmq_proxy since libzmq 3, in which case the first > argument is actually ignored. > > This can be fixed by making the REP socket a ROUTER, which is what it > should be anyway. There?s one caveat for this change, though: On very old > systems with libzmq-2, ROUTERs don?t work in a FORWARDER device, only > QUEUE. This is fine, and doesn?t really make a difference. > > To be fully compatible with libzmq-2.2 through 4.1, the heartbeat call > should change from: > > zmq_device(FORWARDER, REP, REP) > > to: > > zmq_device(QUEUE, ROUTER, ROUTER) > > If you only need to support libzmq ? 3 (recommended), you can use: > > zmq_proxy(ROUTER, ROUTER) > > If you were using your own echo loop for the heartbeat socket instead of > zmq_device, you shouldn?t be affected. > > This is really an implementation detail, not a protocol change, because it > doesn?t affect how messages are sent or received by the kernel, only the > internal event handling (REP is a restricted subclass of ROUTER, > indistinguishable to peers). > > -MinRK > ? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritemio at gmail.com Thu Jun 11 18:58:00 2015 From: tritemio at gmail.com (Antonino Ingargiola) Date: Thu, 11 Jun 2015 15:58:00 -0700 Subject: [IPython-dev] Jupyter/IPython for beginner Message-ID: Hi to all, lately I have written a few notebooks that I tend to "see" as self-contained quasi-graphical application that even users with no programming experience can reasonably use. There are use cases in which a notebook can be used by just changing the input file name or tweaking some parameter. The widgets are powerful addons to include some more advanced interactivity. The only problem with this vision is that all the needed software is still not exactly straightforward to install (mostly on Windows and OSX), and running the Jupyter Notebook app can be confusing (think of users that have never used a terminal, explain the start-up folder issue, etc...). conda and Anaconda are great tools but there a still quite a few steps compared to a click and run application. So,I'd like to raise two issue/question. #1 I'm trying to collect some "absolute beginner" friendly documentation to go from 0 to "knowing how to start, stop and run a notebook on a local machine". The only resource that I know is the OpenTechSchool tutorial[1] (and I, for one, also wrote a short tutorial[2]). But this kind of documentation with all the practical steps spelled out tends to become outdated pretty quickly. So maybe, could be useful to build a concise page like "Jupyther Notebook for the absolute beginner" (or something like that) that will help novice users to get started, that is prominently advertised and that is easy to maintain (i.e. short). #2, except for the documentation, has anybody ever worked on a "starter app" for notebooks. Something between pyinstaller and an ipython launcher. If you think of a notebook as an app (not as a document) it makes sense that the notebook shows up with a double click, without installing anything else. In this case the notebook may be read-only by default. Any pointers, ideas? Antonio -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Thu Jun 11 20:47:29 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 11 Jun 2015 17:47:29 -0700 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: References: Message-ID: On 11 June 2015 at 15:58, Antonino Ingargiola wrote: > except for the documentation, has anybody ever worked on a "starter app" > for notebooks. Something between pyinstaller and an ipython launcher. If > you think of a notebook as an app (not as a document) it makes sense that > the notebook shows up with a double click, without installing anything else I have a project called nbopen, which can set things up so that double clicking on a notebook opens it. It doesn't make it any easier to get everything you need installed, though. https://github.com/takluyver/nbopen Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick.bollweg at gmail.com Thu Jun 11 23:13:41 2015 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Fri, 12 Jun 2015 03:13:41 +0000 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: References: Message-ID: Mentioned frequently, I think hands-down the easiest way to install all the things for the beginner, especially on windows, is anaconda... it installs almost 200 vetted packages into userland (no permissions issues) and uses binaries (no compiler toolchain needed). it comes with a graphical launcher for the notebook: While it doesn't ship with the notebook, it's one click away, and I'd still say it is about the simplest you can get. nbopen seems like it would be a nice addition: i would say boutique, nb-specific things would be out of scope, but ana ships runipy, so maybe the case could be made for inclusion. Of course, there are the hosted solutions: sagemathcloud and tmpnb, depending on your needs. Going beyond the above towards single-notebook app-ification is interesting. Some custom version of atom/hydrogen with an embedded anaconda environment would be a remarkable installable experience. On Thu, Jun 11, 2015 at 8:48 PM Thomas Kluyver wrote: > On 11 June 2015 at 15:58, Antonino Ingargiola wrote: > >> except for the documentation, has anybody ever worked on a "starter app" >> for notebooks. Something between pyinstaller and an ipython launcher. If >> you think of a notebook as an app (not as a document) it makes sense that >> the notebook shows up with a double click, without installing anything else > > > I have a project called nbopen, which can set things up so that double > clicking on a notebook opens it. It doesn't make it any easier to get > everything you need installed, though. > > https://github.com/takluyver/nbopen > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: new-launcher.png Type: image/png Size: 107778 bytes Desc: not available URL: From wstein at gmail.com Fri Jun 12 00:53:40 2015 From: wstein at gmail.com (William Stein) Date: Thu, 11 Jun 2015 21:53:40 -0700 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: References: Message-ID: On Thursday, June 11, 2015, Nicholas Bollweg wrote: > Mentioned frequently, I think hands-down the easiest way to install all > the things for the beginner, especially on windows, is anaconda... it > installs almost 200 vetted packages into userland (no permissions issues) > and uses binaries (no compiler toolchain needed). > > it comes with a graphical launcher for the notebook: > > While it doesn't ship with the notebook, it's one click away, and I'd > still say it is about the simplest you can get. nbopen seems like it would > be a nice addition: i would say boutique, nb-specific things would be out > of scope, but ana ships runipy, so maybe the case could be made for > inclusion. > > Of course, there are the hosted solutions: sagemathcloud and tmpnb, > depending on your needs. > And wakari.io and sandstorm.io... > > Going beyond the above towards single-notebook app-ification is > interesting. Some custom version of atom/hydrogen with an embedded anaconda > environment would be a remarkable installable experience. > > On Thu, Jun 11, 2015 at 8:48 PM Thomas Kluyver > wrote: > >> On 11 June 2015 at 15:58, Antonino Ingargiola > > wrote: >> >>> except for the documentation, has anybody ever worked on a "starter app" >>> for notebooks. Something between pyinstaller and an ipython launcher. If >>> you think of a notebook as an app (not as a document) it makes sense that >>> the notebook shows up with a double click, without installing anything else >> >> >> I have a project called nbopen, which can set things up so that double >> clicking on a notebook opens it. It doesn't make it any easier to get >> everything you need installed, though. >> >> https://github.com/takluyver/nbopen >> >> Thomas >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > -- Sent from my massive iPhone 6 plus. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: new-launcher.png Type: image/png Size: 107778 bytes Desc: not available URL: From steve at holdenweb.com Fri Jun 12 02:40:55 2015 From: steve at holdenweb.com (Steve Holden) Date: Fri, 12 Jun 2015 07:40:55 +0100 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: References: Message-ID: <8BEDC80B-F92E-46F9-8F5B-9E7C0B4D320B@holdenweb.com> I recently got so enthusiastic about anaconda I prepared a short video outlining the install. This demonstrates how easy it is. http://holdenweb.blogspot.co.uk/2015/02/how-to-get-almost-all-python-you-might.html. For those wanting a tailored configuration there's miniconda, too, and that's detailed at http://holdenweb.blogspot.co.uk/2015/02/how-to-get-bits-of-python-you-need.html. regards Steve On Jun 12, 2015, at 5:53 AM, William Stein wrote: > > > On Thursday, June 11, 2015, Nicholas Bollweg wrote: > Mentioned frequently, I think hands-down the easiest way to install all the things for the beginner, especially on windows, is anaconda... it installs almost 200 vetted packages into userland (no permissions issues) and uses binaries (no compiler toolchain needed). > > it comes with a graphical launcher for the notebook: > > While it doesn't ship with the notebook, it's one click away, and I'd still say it is about the simplest you can get. nbopen seems like it would be a nice addition: i would say boutique, nb-specific things would be out of scope, but ana ships runipy, so maybe the case could be made for inclusion. > > Of course, there are the hosted solutions: sagemathcloud and tmpnb, depending on your needs. > > And wakari.io and sandstorm.io... > > > > Going beyond the above towards single-notebook app-ification is interesting. Some custom version of atom/hydrogen with an embedded anaconda environment would be a remarkable installable experience. > > On Thu, Jun 11, 2015 at 8:48 PM Thomas Kluyver wrote: > On 11 June 2015 at 15:58, Antonino Ingargiola wrote: > except for the documentation, has anybody ever worked on a "starter app" for notebooks. Something between pyinstaller and an ipython launcher. If you think of a notebook as an app (not as a document) it makes sense that the notebook shows up with a double click, without installing anything else > > I have a project called nbopen, which can set things up so that double clicking on a notebook opens it. It doesn't make it any easier to get everything you need installed, though. > > https://github.com/takluyver/nbopen > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > -- > Sent from my massive iPhone 6 plus. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / @holdenweb -------------- next part -------------- An HTML attachment was scrubbed... URL: From bblais at gmail.com Fri Jun 12 07:08:40 2015 From: bblais at gmail.com (Brian Blais) Date: Fri, 12 Jun 2015 07:08:40 -0400 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: <8BEDC80B-F92E-46F9-8F5B-9E7C0B4D320B@holdenweb.com> References: <8BEDC80B-F92E-46F9-8F5B-9E7C0B4D320B@holdenweb.com> Message-ID: The problem is with workflow, I find. I use the IPython Notebook to introduce programming, and run into this problem all the time, and it does cause confusion amongst newbies. The problem with the anaconda launcher is that it opens the notebook in a default directory. I don't know about others, but I usually organize my projects by folders, and don't want to point to a default directory - I want the notebook to open in *this* directory (the one with the notebook file in it). I also don't want to have to navigate in the launcher, I'd much rather navigate with the OS. Thus, a double-click open would be great. I haven't checked nbopen (does it check to see if a server is already running in that folder, etc..?) but it looks good. I usually ship a .bat file or .command file with something as simple as: @echo off ipython notebook so that it starts in that folder. It's still not as intuitive as double-clicking the .ipynb file, which I think is the ideal. bb On Fri, Jun 12, 2015 at 2:40 AM, Steve Holden wrote: > I recently got so enthusiastic about anaconda I prepared a short video > outlining the install. This demonstrates how easy it is. > http://holdenweb.blogspot.co.uk/2015/02/how-to-get-almost-all-python-you-might.html. > For those wanting a tailored configuration there's miniconda, too, and > that's detailed at > http://holdenweb.blogspot.co.uk/2015/02/how-to-get-bits-of-python-you-need.html. > > regards > Steve > > On Jun 12, 2015, at 5:53 AM, William Stein wrote: > > > > On Thursday, June 11, 2015, Nicholas Bollweg wrote: >> >> Mentioned frequently, I think hands-down the easiest way to install all >> the things for the beginner, especially on windows, is anaconda... it >> installs almost 200 vetted packages into userland (no permissions issues) >> and uses binaries (no compiler toolchain needed). >> >> it comes with a graphical launcher for the notebook: >> >> While it doesn't ship with the notebook, it's one click away, and I'd >> still say it is about the simplest you can get. nbopen seems like it would >> be a nice addition: i would say boutique, nb-specific things would be out of >> scope, but ana ships runipy, so maybe the case could be made for inclusion. >> >> Of course, there are the hosted solutions: sagemathcloud and tmpnb, >> depending on your needs. > > > And wakari.io and sandstorm.io... > > >> >> >> Going beyond the above towards single-notebook app-ification is >> interesting. Some custom version of atom/hydrogen with an embedded anaconda >> environment would be a remarkable installable experience. >> >> On Thu, Jun 11, 2015 at 8:48 PM Thomas Kluyver wrote: >>> >>> On 11 June 2015 at 15:58, Antonino Ingargiola wrote: >>>> >>>> except for the documentation, has anybody ever worked on a "starter app" >>>> for notebooks. Something between pyinstaller and an ipython launcher. If you >>>> think of a notebook as an app (not as a document) it makes sense that the >>>> notebook shows up with a double click, without installing anything else >>> >>> >>> I have a project called nbopen, which can set things up so that double >>> clicking on a notebook opens it. It doesn't make it any easier to get >>> everything you need installed, though. >>> >>> https://github.com/takluyver/nbopen >>> >>> Thomas >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > -- > Sent from my massive iPhone 6 plus. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > -- > Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / > @holdenweb > > > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- ----------------- bblais at gmail.com http://web.bryant.edu/~bblais From nick.bollweg at gmail.com Fri Jun 12 09:08:40 2015 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Fri, 12 Jun 2015 13:08:40 +0000 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: References: <8BEDC80B-F92E-46F9-8F5B-9E7C0B4D320B@holdenweb.com> Message-ID: Ah, too bad. Maybe this is an argument for including nbopen, as who needs another file browser? Even more conda-y would be to somehow combine with: https://github.com/chdoig/conda-auto-env Double click to not only start a notebook server, but also (create and) activate a custom environment. On 07:08, Fri, Jun 12, 2015 Brian Blais wrote: > The problem is with workflow, I find. I use the IPython Notebook to > introduce programming, and run into this problem all the time, and it > does cause confusion amongst newbies. The problem with the anaconda > launcher is that it opens the notebook in a default directory. I > don't know about others, but I usually organize my projects by > folders, and don't want to point to a default directory - I want the > notebook to open in *this* directory (the one with the notebook file > in it). I also don't want to have to navigate in the launcher, I'd > much rather navigate with the OS. Thus, a double-click open would be > great. I haven't checked nbopen (does it check to see if a server is > already running in that folder, etc..?) but it looks good. I usually > ship a .bat file or .command file with something as simple as: > > @echo off > ipython notebook > > so that it starts in that folder. It's still not as intuitive as > double-clicking the .ipynb file, which I think is the ideal. > > bb > > > > On Fri, Jun 12, 2015 at 2:40 AM, Steve Holden wrote: > > I recently got so enthusiastic about anaconda I prepared a short video > > outlining the install. This demonstrates how easy it is. > > > http://holdenweb.blogspot.co.uk/2015/02/how-to-get-almost-all-python-you-might.html > . > > For those wanting a tailored configuration there's miniconda, too, and > > that's detailed at > > > http://holdenweb.blogspot.co.uk/2015/02/how-to-get-bits-of-python-you-need.html > . > > > > regards > > Steve > > > > On Jun 12, 2015, at 5:53 AM, William Stein wrote: > > > > > > > > On Thursday, June 11, 2015, Nicholas Bollweg > wrote: > >> > >> Mentioned frequently, I think hands-down the easiest way to install all > >> the things for the beginner, especially on windows, is anaconda... it > >> installs almost 200 vetted packages into userland (no permissions > issues) > >> and uses binaries (no compiler toolchain needed). > >> > >> it comes with a graphical launcher for the notebook: > >> > >> While it doesn't ship with the notebook, it's one click away, and I'd > >> still say it is about the simplest you can get. nbopen seems like it > would > >> be a nice addition: i would say boutique, nb-specific things would be > out of > >> scope, but ana ships runipy, so maybe the case could be made for > inclusion. > >> > >> Of course, there are the hosted solutions: sagemathcloud and tmpnb, > >> depending on your needs. > > > > > > And wakari.io and sandstorm.io... > > > > > >> > >> > >> Going beyond the above towards single-notebook app-ification is > >> interesting. Some custom version of atom/hydrogen with an embedded > anaconda > >> environment would be a remarkable installable experience. > >> > >> On Thu, Jun 11, 2015 at 8:48 PM Thomas Kluyver > wrote: > >>> > >>> On 11 June 2015 at 15:58, Antonino Ingargiola > wrote: > >>>> > >>>> except for the documentation, has anybody ever worked on a "starter > app" > >>>> for notebooks. Something between pyinstaller and an ipython launcher. > If you > >>>> think of a notebook as an app (not as a document) it makes sense that > the > >>>> notebook shows up with a double click, without installing anything > else > >>> > >>> > >>> I have a project called nbopen, which can set things up so that double > >>> clicking on a notebook opens it. It doesn't make it any easier to get > >>> everything you need installed, though. > >>> > >>> https://github.com/takluyver/nbopen > >>> > >>> Thomas > >>> _______________________________________________ > >>> IPython-dev mailing list > >>> IPython-dev at scipy.org > >>> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > -- > > Sent from my massive iPhone 6 plus. > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > > Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / > > @holdenweb > > > > > > > > > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > ----------------- > > bblais at gmail.com > http://web.bryant.edu/~bblais > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Fri Jun 12 12:21:15 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 12 Jun 2015 09:21:15 -0700 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: References: <8BEDC80B-F92E-46F9-8F5B-9E7C0B4D320B@holdenweb.com> Message-ID: On 12 June 2015 at 04:08, Brian Blais wrote: > I haven't checked nbopen (does it check to see if a server is > already running in that folder, etc..?) > Yep! It actually checks for a server running in that folder or any parent folder, and uses the one closest one. And of course it starts a new server if there isn't a suitable one already. Nicholas > nbopen seems like it would be a nice addition [to Anaconda] I think so, but the file manager integration probably can't be done just by adding a conda package for nbopen, because it requires setting things up on the system outside the directory managed by conda. The Anaconda installer could set it up, if Continuum want to do that. We can try talking to them about it. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritemio at gmail.com Sat Jun 13 18:42:44 2015 From: tritemio at gmail.com (Antonino Ingargiola) Date: Sat, 13 Jun 2015 15:42:44 -0700 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: References: <8BEDC80B-F92E-46F9-8F5B-9E7C0B4D320B@holdenweb.com> Message-ID: Thanks for all your inputs. Seems like a common page for a quick-start guide for absolute beginners is a good idea and can save many "customized" explanations for many project providing "ready to run" notebooks. So I started this page: http://jupyter-notebook-beginner-guide.readthedocs.org/ Hopefully we can share this among many project and remove duplication of efforts and "confusion" for new users. Suggestion for best resources to link and contributions are welcome. Antonio On Fri, Jun 12, 2015 at 9:21 AM, Thomas Kluyver wrote: > On 12 June 2015 at 04:08, Brian Blais wrote: > >> I haven't checked nbopen (does it check to see if a server is >> already running in that folder, etc..?) >> > > Yep! It actually checks for a server running in that folder or any parent > folder, and uses the one closest one. And of course it starts a new server > if there isn't a suitable one already. > > Nicholas > > nbopen seems like it would be a nice addition [to Anaconda] > > I think so, but the file manager integration probably can't be done just > by adding a conda package for nbopen, because it requires setting things up > on the system outside the directory managed by conda. The Anaconda > installer could set it up, if Continuum want to do that. We can try talking > to them about it. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Sat Jun 13 20:46:02 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Sat, 13 Jun 2015 17:46:02 -0700 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: References: <8BEDC80B-F92E-46F9-8F5B-9E7C0B4D320B@holdenweb.com> Message-ID: <1A708E19-C88F-4CD4-810B-8A61ED2BCFA7@gmail.com> > On Jun 13, 2015, at 15:42, Antonino Ingargiola wrote: > > Thanks for all your inputs. > > Seems like a common page for a quick-start guide for absolute beginners is a good idea and can save many "customized" explanations for many project providing "ready to run" notebooks. > > So I started this page: > > http://jupyter-notebook-beginner-guide.readthedocs.org/ > > Hopefully we can share this among many project and remove duplication of efforts and "confusion" for new users. > > Suggestion for best resources to link and contributions are welcome. Awesome, this is typically the kind of things we want in the **official** docs. Brian and john are working on something like this here: https://github.com/jupyter/jupyter http://jupyter.readthedocs.org/en/latest/ I think combining the effort would be nice. ? M -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritemio at gmail.com Sun Jun 14 13:26:23 2015 From: tritemio at gmail.com (Antonino Ingargiola) Date: Sun, 14 Jun 2015 10:26:23 -0700 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: <1A708E19-C88F-4CD4-810B-8A61ED2BCFA7@gmail.com> References: <8BEDC80B-F92E-46F9-8F5B-9E7C0B4D320B@holdenweb.com> <1A708E19-C88F-4CD4-810B-8A61ED2BCFA7@gmail.com> Message-ID: On Sat, Jun 13, 2015 at 5:46 PM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > > On Jun 13, 2015, at 15:42, Antonino Ingargiola wrote: > > > http://jupyter-notebook-beginner-guide.readthedocs.org/ > > > Awesome, this is typically the kind of things we want in the **official** > docs. > Absolutely, I started that page only because I couldn't find a similar resorce in the official docs. > Brian and john are working on something like this here: > > https://github.com/jupyter/jupyter > > http://jupyter.readthedocs.org/en/latest/ > > I think combining the effort would be nice. > Great. As a step 0, I would add a link to the readthedocs website in the README. I wasn't aware of this repo BTW. I guess we want to discuss a little where and what to add regarding this "beginner quick-start" guide. What are the plan for the docs? They will be all sphinx-based in single stand-alone repository? There will be another repo for the main website? Bottom line, tell me how to contribute and I will adapt that pages. Also a section "how to contribute to the docs" would be usefull ;). Antonio -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Sun Jun 14 16:16:33 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Sun, 14 Jun 2015 13:16:33 -0700 Subject: [IPython-dev] Jupyter/IPython for beginner In-Reply-To: References: <8BEDC80B-F92E-46F9-8F5B-9E7C0B4D320B@holdenweb.com> <1A708E19-C88F-4CD4-810B-8A61ED2BCFA7@gmail.com> Message-ID: > Awesome, this is typically the kind of things we want in the **official** docs. > > Absolutely, I started that page only because I couldn't find a similar resorce in the official docs. Yes we are bad at docs, and also at documenting the docs :-) > > Brian and john are working on something like this here: > > https://github.com/jupyter/jupyter > > http://jupyter.readthedocs.org/en/latest/ > > I think combining the effort would be nice. > > Great. As a step 0, I would add a link to the readthedocs website in the README. I wasn't aware of this repo BTW. That?s because the repo is 2 weeks old, and we are still figuring things out. > I guess we want to discuss a little where and what to add regarding this "beginner quick-start" guide. Sure, the discussion can take place on the repo itself. We can have a section beginners guide I guess. > What are the plan for the docs? They will be all sphinx-based in single stand-alone repository? There will be another repo for the main website? I?ll try to recap a bit, I might be not perfectly correct and there might be technical limitation that can make this a bit different. So jupyter/jupyter should be high level docs, Like first page of the docs, and mainly something that redirect you toward each project respective docs. It should also get some high level how to use/examples, maybe a beginner docs. It should get a section ?how to get help?, ?how are the docs structured? And have link to sub documentation. For technical reason, many sub-doc are not in this repo. Qtconsole docs is in jupyter/qtconsole, IPython only docs in IPython/IPython repo, R kernel doc in IRkernel/IRkernel. It?s ok to put a short description on link to each of these in jupyter/jupyter. The Main Jupyter website (jupyter.org ) is on github://jupyter/jupyter.github.io [1]. The Main IPython website (ipython.org ) is on github://ipython/ipython-website The docs a re mostly sphinx based, we plan on integrating notebooks. \ > Bottom line, tell me how to contribute and I will adapt that pages. Also a section "how to contribute to the docs" would be usefull ;). I would go on Jupyter/Jupyter create an issue there. Brian (and Jon which should be close to LA) are the ones working on that, and are most likely to be aware of the last changes, and how they plan to organise things. They are most likely not reading IPython-dev ML, so haven?t pitched in. ? M [1]: this should definitively be a valid url. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkpaustin at gmail.com Sun Jun 14 16:31:30 2015 From: mkpaustin at gmail.com (taleb brost) Date: Sun, 14 Jun 2015 13:31:30 -0700 Subject: [IPython-dev] Moving IPython javascript methods from cells to custom.js Message-ID: I would like to make two keyboard shortcuts posted in http://stackoverflow.com/questions/28309430/edit-ipython-cell-in-an-external-editor default mappings for all my notebooks. Can someone point me to the correct syntax to append to static/custom.js, given the working cell code below? %%javascript IPython.keyboard_manager.command_shortcuts.add_shortcut('g', { handler : function (event) { var input = IPython.notebook.get_selected_cell().get_text(); var cmd = "f = open('.toto.py', 'w');f.close()"; if (input != "") { cmd = '%%writefile .toto.py\n' + input; } IPython.notebook.kernel.execute(cmd); cmd = "import os;os.system('gvim .toto.py')"; IPython.notebook.kernel.execute(cmd); return false; }} ); IPython.keyboard_manager.command_shortcuts.add_shortcut('u', { handler : function (event) { function handle_output(msg) { var ret = msg.content.text; IPython.notebook.get_selected_cell().set_text(ret); } var callback = {'output': handle_output}; var cmd = "f = open('.toto.py', 'r');print(f.read())"; IPython.notebook.kernel.execute(cmd, {iopub: callback}, {silent: false}); return false; }} ); -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Sun Jun 14 17:10:03 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Sun, 14 Jun 2015 14:10:03 -0700 Subject: [IPython-dev] Moving IPython javascript methods from cells to custom.js In-Reply-To: References: Message-ID: > On Jun 14, 2015, at 13:31, taleb brost wrote: > > I would like to make two keyboard shortcuts posted in > http://stackoverflow.com/questions/28309430/edit-ipython-cell-in-an-external-editor default mappings for all my notebooks. Can someone point > me to the correct syntax to append to static/custom.js, given the > working cell code below? > You probably need to wrap that in a : ``` require([?base/js/events?,?base/js/namespace?], function(events, IPython){ events.on("app_initialized.NotebookApp?, function(){ // your code. } }) [not tested] the outer block, making sure the code is triggered only once events and IPython are available, the inner block making sure this is triggered That being said, I would look at how to write your own extension, that can be activated/deactivated programatically and shared more easily. See following for example. https://github.com/ipython-contrib/IPython-notebook-extensions Which is cleaner. ? M > %%javascript > > IPython.keyboard_manager.command_shortcuts.add_shortcut('g', { > handler : function (event) { > var input = IPython.notebook.get_selected_cell().get_text(); > var cmd = "f = open('.toto.py', 'w');f.close()"; > if (input != "") { > cmd = '%%writefile .toto.py\n' + input; > } > IPython.notebook.kernel.execute(cmd); > cmd = "import os;os.system('gvim .toto.py')"; > IPython.notebook.kernel.execute(cmd); > return false; > }} > ); > > IPython.keyboard_manager.command_shortcuts.add_shortcut('u', { > handler : function (event) { > function handle_output(msg) { > var ret = msg.content.text; > IPython.notebook.get_selected_cell().set_text(ret); > } > var callback = {'output': handle_output}; > var cmd = "f = open('.toto.py', 'r');print(f.read())"; > IPython.notebook.kernel.execute(cmd, {iopub: callback}, {silent: false}); > return false; > }} > ); > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Sun Jun 14 17:22:28 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Sun, 14 Jun 2015 14:22:28 -0700 Subject: [IPython-dev] [jupyter] Fwd: [sage-devel] European Horizon 2020 project OpenDreamKit accepted In-Reply-To: References: <20150519214724.GL14001@mistral> Message-ID: Yes, many congrats on this. Very exciting to see this moving forward and expanding the European side of funding for this ecosystem. Cheers, Brian On Tue, May 19, 2015 at 4:12 PM, Fernando Perez wrote: > Hi all, > > congrats to the OpenDreamKit team, who did an amazing job with an insanely > competitive grant application. We're delighted to be able to collaborate > with them, and look forward to having our EU-based operation grow :) > > Cheers, > > f > > > ---------- Forwarded message ---------- > From: Nicolas M. Thiery > Date: Tue, May 19, 2015 at 2:47 PM > Subject: [sage-devel] European Horizon 2020 project OpenDreamKit accepted > To: sage-devel at googlegroups.com, sage-combinat-devel at googlegroups.com > > > Dear Sage developers, > > We are delighted to announce that the Horizon 2020 research proposal > OpenDreamKit was accepted by the European commission: > > opendreamkit.org > > Starting next Fall and for four years, this project will provide > substantial funding to the open source computational mathematics > ecosystem, and in particular popular tools such as LinBox, MPIR, > SageMath, GAP, Pari/GP, LMFDB, Singular, and the IPython/Jupyter > interactive computing environment. > > The total budget is about 7.6 million euros. The largest portion of > that will be devoted to employing an average of 11 researchers and > developers working full time on the project in Europe. We will > announce job openings in the coming weeks; stay tuned! > > Additionally, the participants will contribute the equivalent of six > other people working full time. Altogether the project involves about > 50 people spread over 15 sites in Europe. > > This is a formidable recognition of the strength and maturity of this > ecosystem, of the power of open source development models, and of the > amazing hard work of many communities over the last decades. > > The writing of the proposal itself was open and collaborative. It grew > out of a reflection on the long term needs of the community. It > benefited considerably from the feedback of many; we would like to > thank all those who helped shape this proposal and make it happen. > > It is our hope that this financial support will help push forward > critical technical tasks. We tried hard in the proposal to make a > worthwhile selection of such tasks, within some constraints imposed by > the specific call. We are now legally committed to treat those tasks > in priority. This kind of long term prediction work is tough: one of > them has actually already been completed by the community in the mean > time! This is great; whenever this will happen we will be able > reprioritize the resources to whatever emerging needs that will arise. > > Ultimately, this project belongs to the community. Get involved! > > Cheers, > Nicolas > -- > Nicolas M. Thi?ry "Isil" > http://Nicolas.Thiery.name/ > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscribe at googlegroups.com. > To post to this group, send email to sage-devel at googlegroups.com. > Visit this group at http://groups.google.com/group/sage-devel. > For more options, visit https://groups.google.com/d/optout. > > > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > -- > You received this message because you are subscribed to the Google Groups > "Project Jupyter" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jupyter+unsubscribe at googlegroups.com. > To post to this group, send email to jupyter at googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jupyter/CAHAreOqGkVpMm%2Buapmwq3-cAZEMPUVem68wDEh6zvwYz4WTCUA%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From mkpaustin at gmail.com Mon Jun 15 18:23:09 2015 From: mkpaustin at gmail.com (Phil Austin) Date: Mon, 15 Jun 2015 15:23:09 -0700 Subject: [IPython-dev] Moving IPython javascript methods from cells to custom.js In-Reply-To: References: Message-ID: On Sun, Jun 14, 2015 at 2:10 PM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > > > That being said, I would look at how to write your own extension, that can > be activated/deactivated programatically and shared more easily. > > See following for example. > > https://github.com/ipython-contrib/IPython-notebook-extensions > Thanks -- in case anyone else is interested, here's an nbextension based on navigation-hotkeys that's working for me https://github.com/phaustin/IPython-notebook-extensions/blob/b8748cd34ef0369c5f03b5035dc40932cb42a8ce/usability/navigation-editor/main.js -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.voorhies at ucsf.edu Thu Jun 18 12:44:19 2015 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Thu, 18 Jun 2015 09:44:19 -0700 Subject: [IPython-dev] symlinks vs. hard links for topic-oriented notebook directories Message-ID: <5582F563.8050508@ucsf.edu> I'm finally moving from one big git-controlled notebook directory to a more organized tree structure. Frequently, I have notebooks that could be classified in more than one subdirectory; e.g., a PCA analysis of an expression profiling experiment might make sense in both the subdirectory for that experiment as well as a subdirectory of PCA analyses. Assuming the IPython working directory is not relevant (e.g., the notebook refers to all data by absolute path), are there any pros/cons (with respect to git and/or IPython) to solving this problem by either symlinking or hard linking such notebooks to additional subdirectories? (Or has anyone come up with a cleaner solution to this problem?) Thanks, Mark From takowl at gmail.com Thu Jun 18 12:54:39 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 18 Jun 2015 09:54:39 -0700 Subject: [IPython-dev] symlinks vs. hard links for topic-oriented notebook directories In-Reply-To: <5582F563.8050508@ucsf.edu> References: <5582F563.8050508@ucsf.edu> Message-ID: On 18 June 2015 at 09:44, Mark Voorhies wrote: > are there any pros/cons > (with respect to git and/or IPython) to solving this problem by either > symlinking or hard linking > such notebooks to additional subdirectories? > git knows about symlinks and should be able to recreate them, but if you want anyone to be able to clone your repository on Windows, it's best to steer clear. http://stackoverflow.com/questions/5917249/git-symlinks-in-windows git does not know about hard links, so it will probably break them at some point. http://stackoverflow.com/questions/3729278/git-and-hard-links Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Thu Jun 18 12:55:46 2015 From: steve at holdenweb.com (Steve Holden) Date: Thu, 18 Jun 2015 17:55:46 +0100 Subject: [IPython-dev] symlinks vs. hard links for topic-oriented notebook directories In-Reply-To: <5582F563.8050508@ucsf.edu> References: <5582F563.8050508@ucsf.edu> Message-ID: <9E480CD0-287C-4D85-B7EB-ED28F57C595C@holdenweb.com> It seems like a reasonable practice to me. While you are at it, think about getting rid of those absolute data file pathnames by adding appropriate symbolic links to a central data directory, then you can move the whole tree about. regards Steve On Jun 18, 2015, at 5:44 PM, Mark Voorhies wrote: > I'm finally moving from one big git-controlled notebook directory to a more organized tree structure. > Frequently, I have notebooks that could be classified in more than one subdirectory; e.g., a PCA > analysis of an expression profiling experiment might make sense in both the subdirectory for that > experiment as well as a subdirectory of PCA analyses. Assuming the IPython working directory is > not relevant (e.g., the notebook refers to all data by absolute path), are there any pros/cons > (with respect to git and/or IPython) to solving this problem by either symlinking or hard linking > such notebooks to additional subdirectories? (Or has anyone come up with a cleaner solution to > this problem?) > > Thanks, > > Mark > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / @holdenweb -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Thu Jun 18 12:59:43 2015 From: steve at holdenweb.com (Steve Holden) Date: Thu, 18 Jun 2015 17:59:43 +0100 Subject: [IPython-dev] symlinks vs. hard links for topic-oriented notebook directories In-Reply-To: References: <5582F563.8050508@ucsf.edu> Message-ID: <43A13D2E-A160-4A42-BF1C-11347AD8494C@holdenweb.com> Gah, completely forgot about Microsoft platforms, sorry. Good catch, Thomas. S On Jun 18, 2015, at 5:54 PM, Thomas Kluyver wrote: > if you want anyone to be able to clone your repository on Windows, it's best to steer clear -- Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / @holdenweb -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick.bollweg at gmail.com Thu Jun 18 13:51:28 2015 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Thu, 18 Jun 2015 17:51:28 +0000 Subject: [IPython-dev] symlinks vs. hard links for topic-oriented notebook directories In-Reply-To: <43A13D2E-A160-4A42-BF1C-11347AD8494C@holdenweb.com> References: <5582F563.8050508@ucsf.edu> <43A13D2E-A160-4A42-BF1C-11347AD8494C@holdenweb.com> Message-ID: For public dissemination, the file tree is probably the best metaphor, with 1:1 correspondence, and we'll just have to deal with your chosen strict hierarchy. However, i could imagine a contentsmanager that worked with metadata to offer a personal workflow like gmail, offering non-mutually exclusive tags or labels and/or the canonical file representation. Downstream apps like nbviewer would probably not implement it: we have enough trouble serving the things you _do_ ask for. On 12:59, Thu, Jun 18, 2015 Steve Holden wrote: > Gah, completely forgot about Microsoft platforms, sorry. Good catch, > Thomas. S > > On Jun 18, 2015, at 5:54 PM, Thomas Kluyver wrote: > > if you want anyone to be able to clone your repository on Windows, it's > best to steer clear > > > -- > Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 > / @holdenweb > > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Thu Jun 18 14:12:07 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Thu, 18 Jun 2015 20:12:07 +0200 Subject: [IPython-dev] symlinks vs. hard links for topic-oriented notebook directories In-Reply-To: References: <5582F563.8050508@ucsf.edu> <43A13D2E-A160-4A42-BF1C-11347AD8494C@holdenweb.com> Message-ID: Notebook atomic save might not play nice neither with symlink nor hardlink. Sorry short from my phone. -- M > On 18 juin 2015, at 19:51, Nicholas Bollweg wrote: > > For public dissemination, the file tree is probably the best metaphor, with 1:1 correspondence, and we'll just have to deal with your chosen strict hierarchy. > > However, i could imagine a contentsmanager that worked with metadata to offer a personal workflow like gmail, offering non-mutually exclusive tags or labels and/or the canonical file representation. > > Downstream apps like nbviewer would probably not implement it: we have enough trouble serving the things you _do_ ask for. > > >> On 12:59, Thu, Jun 18, 2015 Steve Holden wrote: >> Gah, completely forgot about Microsoft platforms, sorry. Good catch, Thomas. S >> >>> On Jun 18, 2015, at 5:54 PM, Thomas Kluyver wrote: >>> >>> if you want anyone to be able to clone your repository on Windows, it's best to steer clear >> >> -- >> Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / @holdenweb >> >> >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Thu Jun 18 14:22:42 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 18 Jun 2015 11:22:42 -0700 Subject: [IPython-dev] symlinks vs. hard links for topic-oriented notebook directories In-Reply-To: References: <5582F563.8050508@ucsf.edu> <43A13D2E-A160-4A42-BF1C-11347AD8494C@holdenweb.com> Message-ID: On 18 June 2015 at 11:12, Matthias Bussonnier wrote: > Notebook atomic save might not play nice neither with symlink nor hardlink. It should be OK with symlinks. The older implementation breaks hard links; the newer one (coming in 4.0) should be OK in normal circumstances, but it will break the hard link if saving fails, when it puts the copy of the previous state back. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.froger.ml at mailoo.org Thu Jun 18 15:59:15 2015 From: david.froger.ml at mailoo.org (David Froger) Date: Thu, 18 Jun 2015 21:59:15 +0200 Subject: [IPython-dev] parallel ouptut before exception Message-ID: <20150618195915.2665.59196@otis.rocq.inria.fr> Hello, I have a IPython 3.1.0 Python 2.7 notebook containing these 2 cells: from IPython.parallel import Client client = Client(profile="mpi") # cluster was started with 4 procs" %%px --block print "message" raise ValueError, "This is a value error" For each engine the exception is displayed in the notebook, but I can't get the "message". (I tried %pxresult, it displays only the exceptions too). How to display the standard output (and standard error)? Thanks, David From nathan12343 at gmail.com Thu Jun 18 16:01:57 2015 From: nathan12343 at gmail.com (Nathan Goldbaum) Date: Thu, 18 Jun 2015 13:01:57 -0700 Subject: [IPython-dev] parallel ouptut before exception In-Reply-To: <20150618195915.2665.59196@otis.rocq.inria.fr> References: <20150618195915.2665.59196@otis.rocq.inria.fr> Message-ID: Do you get output if you add "sys.stdout.flush()" before raising the ValueError? On Thu, Jun 18, 2015 at 12:59 PM, David Froger wrote: > Hello, > > I have a IPython 3.1.0 Python 2.7 notebook containing these 2 cells: > > from IPython.parallel import Client > client = Client(profile="mpi") # cluster was started with 4 procs" > > %%px --block > print "message" > raise ValueError, "This is a value error" > > For each engine the exception is displayed in the notebook, but I can't > get the > "message". (I tried %pxresult, it displays only the exceptions too). > > How to display the standard output (and standard error)? > > Thanks, > David > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.froger.ml at mailoo.org Thu Jun 18 16:36:54 2015 From: david.froger.ml at mailoo.org (David Froger) Date: Thu, 18 Jun 2015 22:36:54 +0200 Subject: [IPython-dev] parallel ouptut before exception In-Reply-To: References: <20150618195915.2665.59196@otis.rocq.inria.fr> Message-ID: <20150618203654.2665.4495@otis.rocq.inria.fr> Hello Nathan, Quoting Nathan Goldbaum (2015-06-18 22:01:57) > Do you get output if you add "sys.stdout.flush()" before raising the > ValueError? Thanks for the suggestion. No, I don't get the output with sys.stdout.flush(). From benjaminrk at gmail.com Thu Jun 18 17:24:56 2015 From: benjaminrk at gmail.com (MinRK) Date: Thu, 18 Jun 2015 14:24:56 -0700 Subject: [IPython-dev] parallel ouptut before exception In-Reply-To: <20150618203654.2665.4495@otis.rocq.inria.fr> References: <20150618195915.2665.59196@otis.rocq.inria.fr> <20150618203654.2665.4495@otis.rocq.inria.fr> Message-ID: There is a way to do this, but it's not provided by IPython by default. The reason being that right now, IPython collates output according to which engine it comes from, so it can't provide incremental output. You can stream outputs as they arrive by watching the metadata, you just want to be sure to identify which engine everything is coming from as they arrive. -MinRK On Thu, Jun 18, 2015 at 1:36 PM, David Froger wrote: > Hello Nathan, > > Quoting Nathan Goldbaum (2015-06-18 22:01:57) > > Do you get output if you add "sys.stdout.flush()" before raising the > > ValueError? > > Thanks for the suggestion. No, I don't get the output with > sys.stdout.flush(). > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From claresloggett at gmail.com Fri Jun 19 02:57:15 2015 From: claresloggett at gmail.com (Clare Sloggett) Date: Fri, 19 Jun 2015 16:57:15 +1000 Subject: [IPython-dev] Internal port error from Jupyterhub Message-ID: Hi all, Is this mailing list still the right place to ask about Jupyterhub? I'm getting an error that seems strange to me, I'm hoping someone might have insight. It's complaining about port availability, and it really looks like that's the error, but the port looks available. So I'm mystified. Short version: when I run jupyterhub --port 9520 --JupyterHub.hub_port=8500 I get an error ("Failed to bind hub to http://localhost:8500/hub/" and "Cannot assign requested address" - detailed error message is below). Yet, if I use the same port to run python -m SimpleHTTPServer 8500 the SimpleHTTPServer runs happily and is accessible, indicating that the port is actually available. This occurs no matter what port I specify for JupyterHub.hub_port. Here are some more details: I'm trying to install on an Ubuntu 14.04 server. There are other services running on this server, but none of them occupy port 8500 (or 9520) and given that SimpleHTTPServer works I am assuming that the other services are probably not the problem? To install Jupyterhub I am running git clone https://github.com/jupyter/jupyterhub cd jupyterhub/ sudo npm install -g configurable-http-proxy sudo apt-get install python3-pip sudo apt-get install nodejs-legacy sudo pip3 install -r requirements.txt sudo pip3 install . If I run Jupyterhub as root (or otherwise), the complete error I get is: $ sudo jupyterhub --port 9520 --JupyterHub.hub_port=8500 [I 2015-06-19 06:51:17.962 JupyterHub app:522] Loading cookie_secret from /home/ubuntu/install_jupyterhub/jupyterhub/jupyterhub_cookie_secret [W 2015-06-19 06:51:18.010 JupyterHub app:260] Generating CONFIGPROXY_AUTH_TOKEN. Restarting the Hub will require restarting the proxy. Set CONFIGPROXY_AUTH_TOKEN env or JupyterHub.proxy_auth_token config to avoid this message. [I 2015-06-19 06:51:18.020 JupyterHub app:623] Not using whitelist. Any authenticated user will be allowed. [E 2015-06-19 06:51:18.034 JupyterHub app:998] Failed to bind hub to http://localhost:8500/hub/ [E 2015-06-19 06:51:18.034 JupyterHub app:1066] Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/jupyterhub/app.py", line 1064, in launch_instance_async yield self.start() File "/usr/local/lib/python3.4/dist-packages/jupyterhub/app.py", line 996, in start self.http_server.listen(self.hub_port, address=self.hub_ip) File "/usr/local/lib/python3.4/dist-packages/tornado/tcpserver.py", line 126, in listen sockets = bind_sockets(port, address=address) File "/usr/local/lib/python3.4/dist-packages/tornado/netutil.py", line 187, in bind_sockets sock.bind(sockaddr) OSError: [Errno 99] Cannot assign requested address Any help would be very much appreciated! Cheers, Clare -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.froger.ml at mailoo.org Fri Jun 19 05:09:40 2015 From: david.froger.ml at mailoo.org (David Froger) Date: Fri, 19 Jun 2015 11:09:40 +0200 Subject: [IPython-dev] parallel ouptut before exception In-Reply-To: References: <20150618195915.2665.59196@otis.rocq.inria.fr> <20150618203654.2665.4495@otis.rocq.inria.fr> Message-ID: <20150619090940.2665.55783@otis.rocq.inria.fr> > There is a way to do this, but it's not provided by IPython by default. The > reason being that right now, IPython collates output according to which engine > it comes from, so it can't provide incremental output. You can stream outputs > as they arrive by watching the metadata, you just want to be sure to identify > which engine everything is coming from as they arrive. Hello MinRK, Do mean like using AsyncResult, as you show in this gist? http://nbviewer.ipython.org/gist/minrk/6171348 Actually, I'm not trying get incremental output, just to get the final output, after running a cell in blocking mode. Sorry if I missunderstood something... For now, the solution I've found is using the logging module with a StreamHandler: file foo.py: import logging def foo(): logger = logging.getLogger('foo') logger.debug("message") raise ValueError, "This is a value error" Notebook: [0] from IPython.parallel import Client client = Client(profile="mpi") # cluster was started with 4 procs [1] %%px --block import logging import StringIO def initialize_logger(name, level): output = StringIO.StringIO() logger = logging.getLogger(name) logger.setLevel(level) stream_handler = logging.StreamHandler(output) stream_handler.setLevel(level) logger.addHandler(stream_handler) return output output = initialize_logger('foo', logging.DEBUG) [2] %%px --block import foo foo.foo() [3] %%px print output.getvalue() From Brian.Chapman at utah.edu Fri Jun 19 13:20:13 2015 From: Brian.Chapman at utah.edu (Brian E Chapman) Date: Fri, 19 Jun 2015 17:20:13 +0000 Subject: [IPython-dev] Nested and blocking calls of Interact or Interactive Message-ID: Hi, A couple of generic questions regarding using Ipython interact or interactive widgets. First, is it possible to use interactive in a nested manner. That is, I have one function that I call with interactive and this function in turn calls another function with interactive? Second, can you call interactive such that execution is blocked until the widget is closed? For example, I would like to call create/execute an interactive widget within a loop. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Fri Jun 19 13:46:05 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 19 Jun 2015 10:46:05 -0700 Subject: [IPython-dev] Internal port error from Jupyterhub In-Reply-To: References: Message-ID: On 18 June 2015 at 23:57, Clare Sloggett wrote: > I get an error ("Failed to bind hub to http://localhost:8500/hub/" and > "Cannot assign requested address" - detailed error message is below). > I wonder if SimpleHTTPServer is using an IP address, and the issue is with the use of the name 'localhost'? -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Fri Jun 19 13:54:13 2015 From: benjaminrk at gmail.com (MinRK) Date: Fri, 19 Jun 2015 10:54:13 -0700 Subject: [IPython-dev] Internal port error from Jupyterhub In-Reply-To: References: Message-ID: On Thu, Jun 18, 2015 at 11:57 PM, Clare Sloggett wrote: Hi all, > > Is this mailing list still the right place to ask about Jupyterhub? > This is fine, but we?re gently directing Jupyter stuff toward the jupyter mailing list . I'm getting an error that seems strange to me, I'm hoping someone might > have insight. It's complaining about port availability, and it really looks > like that's the error, but the port looks available. So I'm mystified. > > Short version: when I run > jupyterhub --port 9520 --JupyterHub.hub_port=8500 > I get an error ("Failed to bind hub to http://localhost:8500/hub/" and > "Cannot assign requested address" - detailed error message is below). > > Yet, if I use the same port to run > python -m SimpleHTTPServer 8500 > the SimpleHTTPServer runs happily and is accessible, indicating that the > port is actually available. > > This occurs no matter what port I specify for JupyterHub.hub_port. > > Here are some more details: > > I'm trying to install on an Ubuntu 14.04 server. There are other services > running on this server, but none of them occupy port 8500 (or 9520) and > given that SimpleHTTPServer works I am assuming that the other services are > probably not the problem? > It?s possible that you can?t bind to localhost, not the port. SimpleHTTPServer binds to all interfaces by default. Try: import socket s = socket.socket() s.bind(('localhost', 8500)) If that fails, you can try 127.0.0.1 instead of localhost. If 127 works, you can set this value in your jupyterhub_config.py file: c.JupyterHub.hub_ip = '127.0.0.1' -MinRK > To install Jupyterhub I am running > > git clone https://github.com/jupyter/jupyterhub > > cd jupyterhub/ > > sudo npm install -g configurable-http-proxy > > sudo apt-get install python3-pip > > sudo apt-get install nodejs-legacy > > sudo pip3 install -r requirements.txt > sudo pip3 install . > > If I run Jupyterhub as root (or otherwise), the complete error I get is: > > $ sudo jupyterhub --port 9520 --JupyterHub.hub_port=8500 > [I 2015-06-19 06:51:17.962 JupyterHub app:522] Loading cookie_secret from > /home/ubuntu/install_jupyterhub/jupyterhub/jupyterhub_cookie_secret > [W 2015-06-19 06:51:18.010 JupyterHub app:260] > Generating CONFIGPROXY_AUTH_TOKEN. Restarting the Hub will require > restarting the proxy. > Set CONFIGPROXY_AUTH_TOKEN env or JupyterHub.proxy_auth_token config > to avoid this message. > > [I 2015-06-19 06:51:18.020 JupyterHub app:623] Not using whitelist. Any > authenticated user will be allowed. > [E 2015-06-19 06:51:18.034 JupyterHub app:998] Failed to bind hub to > http://localhost:8500/hub/ > [E 2015-06-19 06:51:18.034 JupyterHub app:1066] > Traceback (most recent call last): > File "/usr/local/lib/python3.4/dist-packages/jupyterhub/app.py", > line 1064, in launch_instance_async > yield self.start() > File "/usr/local/lib/python3.4/dist-packages/jupyterhub/app.py", > line 996, in start > self.http_server.listen(self.hub_port, address=self.hub_ip) > File "/usr/local/lib/python3.4/dist-packages/tornado/tcpserver.py", > line 126, in listen > sockets = bind_sockets(port, address=address) > File "/usr/local/lib/python3.4/dist-packages/tornado/netutil.py", > line 187, in bind_sockets > sock.bind(sockaddr) > OSError: [Errno 99] Cannot assign requested address > > Any help would be very much appreciated! > > Cheers, > Clare > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Fri Jun 19 14:07:14 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 19 Jun 2015 11:07:14 -0700 Subject: [IPython-dev] Nested and blocking calls of Interact or Interactive In-Reply-To: References: Message-ID: On 19 June 2015 at 10:20, Brian E Chapman wrote: > First, is it possible to use interactive in a nested manner. That is, I > have one function that I call with interactive and this function in turn > calls another function with interactive? > > Second, can you call interactive such that execution is blocked until > the widget is closed? For example, I would like to call create/execute an > interactive widget within a loop. > Widgets can't pause code, and the kernel won't process widget updates while the kernel is running in the main thread. You could probably hack something together using generators and/or threads to make it look like it was doing that, but I wouldn't recommend it. Can you describe at a higher level what you're trying to do - we might be able to suggest other ways to achieve it. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From Brian.Chapman at utah.edu Fri Jun 19 17:00:48 2015 From: Brian.Chapman at utah.edu (Brian E Chapman) Date: Fri, 19 Jun 2015 21:00:48 +0000 Subject: [IPython-dev] Nested and blocking calls of Interact or Interactive In-Reply-To: References: Message-ID: Here in a nutshell is what I?m trying to do. Iterate over (or select from a dropdown widget) a list of directories corresponding to medical imaging exams For the selected directory (exam) select a subdirectory (image series) For the selected series Read in the image interactively define a bounding box in the image using IntRangeSlider Since interactive doesn?t block, I can?t work it into a loop. On the other hand, I haven?t been able to get an interactive widget to generate another interactive widget. Brian From: Thomas Kluyver > Reply-To: IPython developers list > Date: Friday, June 19, 2015 at 12:07 PM To: IPython developers list > Subject: Re: [IPython-dev] Nested and blocking calls of Interact or Interactive On 19 June 2015 at 10:20, Brian E Chapman > wrote: First, is it possible to use interactive in a nested manner. That is, I have one function that I call with interactive and this function in turn calls another function with interactive? Second, can you call interactive such that execution is blocked until the widget is closed? For example, I would like to call create/execute an interactive widget within a loop. Widgets can't pause code, and the kernel won't process widget updates while the kernel is running in the main thread. You could probably hack something together using generators and/or threads to make it look like it was doing that, but I wouldn't recommend it. Can you describe at a higher level what you're trying to do - we might be able to suggest other ways to achieve it. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Fri Jun 19 17:18:27 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 19 Jun 2015 14:18:27 -0700 Subject: [IPython-dev] Nested and blocking calls of Interact or Interactive In-Reply-To: References: Message-ID: On 19 June 2015 at 14:00, Brian E Chapman wrote: > Here in a nutshell is what I?m trying to do. > > Iterate over (or select from a dropdown widget) a list of directories > corresponding to medical imaging exams > For the selected directory (exam) select a subdirectory (image series) > For the selected series > Read in the image > interactively define a bounding box in the image using IntRangeSlider > I think you can do this by dropping down a level from @interactive and using the widget objects directly. Create the set of widgets you need, and when the first one is changed, have it update the available options for the second one. Have a look at these example notebooks: https://github.com/ipython/ipython/blob/3.x/examples/Interactive%20Widgets/Export%20As%20%28nbconvert%29.ipynb https://github.com/ipython/ipython/blob/3.x/examples/Interactive%20Widgets/Widget%20Events.ipynb The widgets don't show up on Github, but if you download them and run them locally, you'll see them. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From Brian.Chapman at utah.edu Fri Jun 19 17:21:50 2015 From: Brian.Chapman at utah.edu (Brian E Chapman) Date: Fri, 19 Jun 2015 21:21:50 +0000 Subject: [IPython-dev] Nested and blocking calls of Interact or Interactive In-Reply-To: References: Message-ID: Thanks. I?ll give it a try. Brian From: Thomas Kluyver > Reply-To: IPython developers list > Date: Friday, June 19, 2015 at 3:18 PM To: IPython developers list > Subject: Re: [IPython-dev] Nested and blocking calls of Interact or Interactive On 19 June 2015 at 14:00, Brian E Chapman > wrote: Here in a nutshell is what I?m trying to do. Iterate over (or select from a dropdown widget) a list of directories corresponding to medical imaging exams For the selected directory (exam) select a subdirectory (image series) For the selected series Read in the image interactively define a bounding box in the image using IntRangeSlider I think you can do this by dropping down a level from @interactive and using the widget objects directly. Create the set of widgets you need, and when the first one is changed, have it update the available options for the second one. Have a look at these example notebooks: https://github.com/ipython/ipython/blob/3.x/examples/Interactive%20Widgets/Export%20As%20%28nbconvert%29.ipynb https://github.com/ipython/ipython/blob/3.x/examples/Interactive%20Widgets/Widget%20Events.ipynb The widgets don't show up on Github, but if you download them and run them locally, you'll see them. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sat Jun 20 14:44:09 2015 From: benjaminrk at gmail.com (MinRK) Date: Sat, 20 Jun 2015 11:44:09 -0700 Subject: [IPython-dev] IPython 3.2 soon Message-ID: We will probably be cutting IPython 3.2 tomorrow, so if anyone knows of any bugfixes already merged that we haven't backported yet to the 3.x branch that should be, open an issue (or better-yet make the patch and issue a PR). Thanks, -MinRK -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sat Jun 20 16:35:18 2015 From: benjaminrk at gmail.com (MinRK) Date: Sat, 20 Jun 2015 13:35:18 -0700 Subject: [IPython-dev] parallel ouptut before exception In-Reply-To: <20150619090940.2665.55783@otis.rocq.inria.fr> References: <20150618195915.2665.59196@otis.rocq.inria.fr> <20150618203654.2665.4495@otis.rocq.inria.fr> <20150619090940.2665.55783@otis.rocq.inria.fr> Message-ID: Ah, in that case it?s a bit easier . I realize that it doesn?t actually make sense to raise exceptions in AsyncResult.display_output(), which would make this much easier. I?ll try to fix that for in 4.0. -MinRK ? On Fri, Jun 19, 2015 at 2:09 AM, David Froger wrote: > > There is a way to do this, but it's not provided by IPython by default. > The > > reason being that right now, IPython collates output according to which > engine > > it comes from, so it can't provide incremental output. You can stream > outputs > > as they arrive by watching the metadata, you just want to be sure to > identify > > which engine everything is coming from as they arrive. > > Hello MinRK, > > Do mean like using AsyncResult, as you show in this gist? > http://nbviewer.ipython.org/gist/minrk/6171348 > > Actually, I'm not trying get incremental output, just to get the final > output, > after running a cell in blocking mode. > > Sorry if I missunderstood something... > > For now, the solution I've found is using the logging module with a > StreamHandler: > > file foo.py: > import logging > def foo(): > logger = logging.getLogger('foo') > logger.debug("message") > raise ValueError, "This is a value error" > > Notebook: > [0] from IPython.parallel import Client > client = Client(profile="mpi") # cluster was started with 4 procs > > [1] %%px --block > import logging > import StringIO > def initialize_logger(name, level): > output = StringIO.StringIO() > logger = logging.getLogger(name) > logger.setLevel(level) > stream_handler = logging.StreamHandler(output) > stream_handler.setLevel(level) > logger.addHandler(stream_handler) > return output > output = initialize_logger('foo', logging.DEBUG) > > [2] %%px --block > import foo > foo.foo() > > [3] %%px > print output.getvalue() > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maximilian.albert at gmail.com Sun Jun 21 18:52:06 2015 From: maximilian.albert at gmail.com (Maximilian Albert) Date: Sun, 21 Jun 2015 23:52:06 +0100 Subject: [IPython-dev] Propagate errors with ipython nbconvert --execute? In-Reply-To: References: Message-ID: Just to follow up on this thread (also for the sake of any future readers): the default behaviour of nbconvert is now to abort if "--execute" is used and any cell raises an error. This behaviour can be overridden with the flag "--allow-errors". See the PR at [1] for the corresponding discussion. Thanks to the IPython/Jupyter team for their helpful comments and reviews! Best wishes, Max [1] https://github.com/jupyter/nbconvert/pull/33 -------------- next part -------------- An HTML attachment was scrubbed... URL: From 36rahu at gmail.com Mon Jun 22 03:27:17 2015 From: 36rahu at gmail.com (Rahul K P) Date: Mon, 22 Jun 2015 12:57:17 +0530 Subject: [IPython-dev] Flask app running interactively on IPython Message-ID: Hi all, Right now am working in image processing in Python. The interface has been build on TKinter at that time we can work the software interactively. The software run using the command run -i program.py. Right now we changed into flask interface and the IPython interactive coding not working here. We didn't get the command line while running. Anyone please help to solve this. Regards Rahul K P -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmckerns at caltech.edu Mon Jun 22 05:50:18 2015 From: mmckerns at caltech.edu (Michael McKerns) Date: Mon, 22 Jun 2015 05:50:18 -0400 Subject: [IPython-dev] drop-in replacement for multiprocessing Message-ID: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> I've put up a drop-in replacement of multiprocessing, that is the entire module code ripped from python, with a few lines edited so that if dill is installed, multiprocessing natively can serialize objects using dill. Essentially, unless you have dill installed, the module does nothing new. It installs as "multiprocessing". Nearly all of python's unit tests pass for all versions of python with it -- I think 4-5 maximum fail (for example, test_unpicklable_object now fails). https://pypi.python.org/pypi/multiprocess/0.70.1 https://github.com/uqfoundation/multiprocess Comments? Good idea, bad idea, otherwise? --- Mike McKerns http://www.its.caltech.edu/~mmckerns From bussonniermatthias at gmail.com Mon Jun 22 07:58:03 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Mon, 22 Jun 2015 13:58:03 +0200 Subject: [IPython-dev] Flask app running interactively on IPython In-Reply-To: References: Message-ID: <16A43FEA-D2DC-4D77-AB38-DB15738885F8@gmail.com> Hi Rahul, > On Jun 22, 2015, at 09:27, Rahul K P <36rahu at gmail.com> wrote: > > Hi all, > Right now am working in image processing in Python. The interface has been build on TKinter at that time we can work the software interactively. The software run using the command run -i program.py. Right now we changed into flask interface and the IPython interactive coding not working here. We didn't get the command line while running. Anyone please help to solve this. We will need a bit more information or code example of what you are doing to be able to help. It would be nice to also understand what you are trying to achieve. Thanks, -- Matthias From rgbkrk at gmail.com Mon Jun 22 13:07:33 2015 From: rgbkrk at gmail.com (Kyle Kelley) Date: Mon, 22 Jun 2015 12:07:33 -0500 Subject: [IPython-dev] IPython 3.2 released, please upgrade Message-ID: Hi everyone, As soon as possible, please upgrade your IPython notebook installation to 3.2. IPython 3.2 contains important security fixes. Users are strongly encouraged to upgrade immediately. Highlights: * A security improvement that sets the secure attribute of the login cookie to prevent them from being sent over http * Revert the face color of matplotlib axes in the inline backend to not be transparent * Enable mathjax safe mode by default * Fix XSS vulnerability in JSON error messages * Various widget-related fixes You can get the latest via pip using `pip install --upgrade ipython[notebook]`. Stay tuned for conda/Anaconda package updates. A CVE has been requested for the XSS vulnerability in http://permalink.gmane.org/gmane.comp.security.oss.general/17131 Thank you to Ahmad Khan of IBM for reporting the XSS vulnerability and Jason Grout for setting MathJax safe mode. -- Kyle Kelley (@rgbkrk ; lambdaops.com, developer.rackspace.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Jun 22 13:33:19 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 22 Jun 2015 10:33:19 -0700 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> References: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> Message-ID: Hi Mike, On 22 June 2015 at 02:50, Michael McKerns wrote: > It installs as "multiprocessing". I'm slightly surprised at this - I'm not sure about shadowing standard library modules. But I can see arguments for it as well, like existing code can benefit from it without needing changes. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Jun 22 13:36:29 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 22 Jun 2015 10:36:29 -0700 Subject: [IPython-dev] Flask app running interactively on IPython In-Reply-To: <16A43FEA-D2DC-4D77-AB38-DB15738885F8@gmail.com> References: <16A43FEA-D2DC-4D77-AB38-DB15738885F8@gmail.com> Message-ID: I assume you mean event loop integration. That's generally only done for GUI event loops, but there's no reason in principle you couldn't integrate with a web app event loop. See the docs here for more info: http://ipython.org/ipython-doc/3/config/eventloops.html (I love having docs for stuff like this - it saves so much explanation when someone asks about it) Thomas On 22 June 2015 at 04:58, Matthias Bussonnier wrote: > Hi Rahul, > > > On Jun 22, 2015, at 09:27, Rahul K P <36rahu at gmail.com> wrote: > > > > Hi all, > > Right now am working in image processing in Python. The interface > has been build on TKinter at that time we can work the software > interactively. The software run using the command run -i program.py. Right > now we changed into flask interface and the IPython interactive coding not > working here. We didn't get the command line while running. Anyone please > help to solve this. > > We will need a bit more information or code example of what you are doing > to be able to help. > It would be nice to also understand what you are trying to achieve. > > Thanks, > -- > Matthias > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Jun 22 13:44:25 2015 From: benjaminrk at gmail.com (MinRK) Date: Mon, 22 Jun 2015 10:44:25 -0700 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: References: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> Message-ID: On Mon, Jun 22, 2015 at 10:33 AM, Thomas Kluyver wrote: Hi Mike, > > On 22 June 2015 at 02:50, Michael McKerns wrote: > >> It installs as "multiprocessing". > > > I'm slightly surprised at this - I'm not sure about shadowing standard > library modules. But I can see arguments for it as well, like existing code > can benefit from it without needing changes. > The evolution of setuptools and pip is making overriding the stdlib harder. I even have a patch proposal in to setuptools and I?m working on one for CPython that would make it basically impossible without a custom PYTHONPATH. For instance, installing this with pip doesn?t result in an importable package, and it can never be distributed as a wheel. As tempting as it is to override the stdlib for ?magic improvements?, I think it?s probably best to avoid namespace collisions with the stdlib, and go with multiprocess or multiprocessing2 or some such. -MinRK > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Jun 22 14:07:25 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 22 Jun 2015 11:07:25 -0700 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: References: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> Message-ID: On 22 June 2015 at 10:44, MinRK wrote: > and go with multiprocess or multiprocessing2 or some such. > I'd suggest something more explicit like multiprocessing_dill. It seems presumptuous call a module 'thing2' without the blessing of the 'thing' maintainers. They might want to use that name one day, like the Python 2.7+ unittest package is backported as unittest2. 'multiprocess' sounds liable to cause confusion - if I saw that and 'multiprocessing' in a tab completion list, I'd probably have to check which one was which. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Jun 22 14:10:47 2015 From: benjaminrk at gmail.com (MinRK) Date: Mon, 22 Jun 2015 11:10:47 -0700 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: References: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> Message-ID: I didn't mean to start a naming discussion, just that I don't think colliding with the stdlib is a good plan. On Mon, Jun 22, 2015 at 11:07 AM, Thomas Kluyver wrote: > On 22 June 2015 at 10:44, MinRK wrote: > >> and go with multiprocess or multiprocessing2 or some such. >> > I'd suggest something more explicit like multiprocessing_dill. It seems > presumptuous call a module 'thing2' without the blessing of the 'thing' > maintainers. They might want to use that name one day, like the Python 2.7+ > unittest package is backported as unittest2. > > 'multiprocess' sounds liable to cause confusion - if I saw that and > 'multiprocessing' in a tab completion list, I'd probably have to check > which one was which. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmckerns at caltech.edu Mon Jun 22 14:42:47 2015 From: mmckerns at caltech.edu (Michael McKerns) Date: Mon, 22 Jun 2015 14:42:47 -0400 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: References: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> Message-ID: <2a52c7303791c2b497db7069f31e8227.squirrel@webmail.caltech.edu> Yeah, basic hesitation in colliding with a stdlib module is the reason I didn't do this a long time ago. The most preferable solution would be to have the underlying "two line patch" accepted to the stdlib that allows a different serializer. I'm not sure if a PEP was submitted for that it would be accepted. But maybe I'd get surprised. The only benefit overriding multiprocessing gives is that all packages that depend on multiprocessing also get accelerated serialization propagated to them. That's really my motivation for making the drop-in replacement. So multiprocessing taking an alternate serializer would work? and it's a simple as dirt patch with apparently no loss of functionality. > I didn't mean to start a naming discussion, just that I don't think > colliding with the stdlib is a good plan. > > On Mon, Jun 22, 2015 at 11:07 AM, Thomas Kluyver wrote: > >> On 22 June 2015 at 10:44, MinRK wrote: >> >>> and go with multiprocess or multiprocessing2 or some such. >>> >> I'd suggest something more explicit like multiprocessing_dill. It seems >> presumptuous call a module 'thing2' without the blessing of the 'thing' >> maintainers. They might want to use that name one day, like the Python >> 2.7+ >> unittest package is backported as unittest2. >> >> 'multiprocess' sounds liable to cause confusion - if I saw that and >> 'multiprocessing' in a tab completion list, I'd probably have to check >> which one was which. >> >> Thomas >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > --- Mike McKerns California Institute of Technology TEL: (626)395-5773 or (626)590-8470 http://www.its.caltech.edu/~mmckerns mmckerns at caltech.edu From benjaminrk at gmail.com Mon Jun 22 14:51:12 2015 From: benjaminrk at gmail.com (MinRK) Date: Mon, 22 Jun 2015 11:51:12 -0700 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: <2a52c7303791c2b497db7069f31e8227.squirrel@webmail.caltech.edu> References: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> <2a52c7303791c2b497db7069f31e8227.squirrel@webmail.caltech.edu> Message-ID: On Mon, Jun 22, 2015 at 11:42 AM, Michael McKerns wrote: Yeah, basic hesitation in colliding with a stdlib module is the reason > I didn't do this a long time ago. The most preferable solution would > be to have the underlying "two line patch" accepted to the stdlib that > allows a different serializer. I'm not sure if a PEP was submitted for > that it would be accepted. But maybe I'd get surprised. > > The only benefit overriding multiprocessing gives is that all packages > that depend on multiprocessing also get accelerated serialization > propagated to them. That's really my motivation for making the > drop-in replacement. So multiprocessing taking an alternate > serializer would work? and it's a simple as dirt patch with > apparently no loss of functionality. > Since it?s adding functionality, people can?t really assume that multiprocessing is your multiprocessing anyway, so they would need a check. That being the case, you could have a two line: import multidill multidill.patch_multiprocessing() If the patch is one that?s possible to make at runtime. Then you have: - library with no collisions - add functionality transparently to multiprocessing, but only explicitly, not implicitly. -MinRK > > > > I didn't mean to start a naming discussion, just that I don't think > > colliding with the stdlib is a good plan. > > > > On Mon, Jun 22, 2015 at 11:07 AM, Thomas Kluyver > wrote: > > > >> On 22 June 2015 at 10:44, MinRK wrote: > >> > >>> and go with multiprocess or multiprocessing2 or some such. > >>> > >> I'd suggest something more explicit like multiprocessing_dill. It seems > >> presumptuous call a module 'thing2' without the blessing of the 'thing' > >> maintainers. They might want to use that name one day, like the Python > >> 2.7+ > >> unittest package is backported as unittest2. > >> > >> 'multiprocess' sounds liable to cause confusion - if I saw that and > >> 'multiprocessing' in a tab completion list, I'd probably have to check > >> which one was which. > >> > >> Thomas > >> > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > >> > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > --- > > Mike McKerns > California Institute of Technology > TEL: (626)395-5773 or (626)590-8470 > http://www.its.caltech.edu/~mmckerns > mmckerns at caltech.edu > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Jun 22 14:51:17 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 22 Jun 2015 11:51:17 -0700 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: <2a52c7303791c2b497db7069f31e8227.squirrel@webmail.caltech.edu> References: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> <2a52c7303791c2b497db7069f31e8227.squirrel@webmail.caltech.edu> Message-ID: On 22 June 2015 at 11:42, Michael McKerns wrote: > The most preferable solution would > be to have the underlying "two line patch" accepted to the stdlib that > allows a different serializer. I'm not sure if a PEP was submitted for > that it would be accepted. But maybe I'd get surprised. > You shouldn't need a PEP for a two line change in a standard library module. File an issue with the patch attached - normally when I've filed issues, someone has responded within a day or two, though sometimes I've needed to prod a bit to keep it on the radar after the initial response. Of course, the downside to that approach is that, as it's a new feature, you won't be able to use it until Python 3.6 (3.5 is already in feature freeze, I believe). But even so, it's good to have advance planning to do things the way you want to. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Mon Jun 22 14:55:06 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 22 Jun 2015 11:55:06 -0700 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: References: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> <2a52c7303791c2b497db7069f31e8227.squirrel@webmail.caltech.edu> Message-ID: I think the best path is to submit a patch to the stdlib - as Thomas mentioned, I would be surprised if a PEP was required for that. Otherwise, a small package that monkey patches the stdlib version is probably best. On Mon, Jun 22, 2015 at 11:51 AM, Thomas Kluyver wrote: > On 22 June 2015 at 11:42, Michael McKerns wrote: >> >> The most preferable solution would >> be to have the underlying "two line patch" accepted to the stdlib that >> allows a different serializer. I'm not sure if a PEP was submitted for >> that it would be accepted. But maybe I'd get surprised. > > > You shouldn't need a PEP for a two line change in a standard library module. > File an issue with the patch attached - normally when I've filed issues, > someone has responded within a day or two, though sometimes I've needed to > prod a bit to keep it on the radar after the initial response. > > Of course, the downside to that approach is that, as it's a new feature, you > won't be able to use it until Python 3.6 (3.5 is already in feature freeze, > I believe). But even so, it's good to have advance planning to do things the > way you want to. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From mmckerns at caltech.edu Mon Jun 22 15:19:18 2015 From: mmckerns at caltech.edu (Michael McKerns) Date: Mon, 22 Jun 2015 15:19:18 -0400 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: References: <88d4d2ee843ce6632f6b08bcbb2295a8.squirrel@webmail.caltech.edu> <2a52c7303791c2b497db7069f31e8227.squirrel@webmail.caltech.edu> Message-ID: <5e8bdc5ce0a1011bb9517a4acf110937.squirrel@webmail.caltech.edu> Yeah? a monkeypach is probably the right way to go... For python 3.3+, it's an actual two-line change that tries to import dill, and failing imports pickle. For python 2.7, there's an additional two-line change in _multiprocessing (the C lib), that does exactly the same as the above. I *think* multiprocessing only imports _multiprocessing as _multiprocessing and not as "multiprocessing._multiprocessing", so a runtime monkeypatch would then still work. The stdlib version is already nearly abstracted for the serializer, so adding 2-3 more lines to pick whatever serializer one would like should also be easy. I'll see about submitting a patch for python 3.6. Thanks all for the feedback. > On Mon, Jun 22, 2015 at 11:42 AM, Michael McKerns > wrote: > > Yeah, basic hesitation in colliding with a stdlib module is the reason >> I didn't do this a long time ago. The most preferable solution would >> be to have the underlying "two line patch" accepted to the stdlib that >> allows a different serializer. I'm not sure if a PEP was submitted for >> that it would be accepted. But maybe I'd get surprised. >> >> The only benefit overriding multiprocessing gives is that all packages >> that depend on multiprocessing also get accelerated serialization >> propagated to them. That's really my motivation for making the >> drop-in replacement. So multiprocessing taking an alternate >> serializer would work? and it's a simple as dirt patch with >> apparently no loss of functionality. >> > Since it?s adding functionality, people can?t really assume that > multiprocessing is your multiprocessing anyway, so they would need a > check. > That being the case, you could have a two line: > > import multidill > multidill.patch_multiprocessing() > > If the patch is one that?s possible to make at runtime. > > Then you have: > > - library with no collisions > - add functionality transparently to multiprocessing, but only > explicitly, not implicitly. > > -MinRK > > >> >> >> > I didn't mean to start a naming discussion, just that I don't think >> > colliding with the stdlib is a good plan. >> > >> > On Mon, Jun 22, 2015 at 11:07 AM, Thomas Kluyver >> wrote: >> > >> >> On 22 June 2015 at 10:44, MinRK wrote: >> >> >> >>> and go with multiprocess or multiprocessing2 or some such. >> >>> >> >> I'd suggest something more explicit like multiprocessing_dill. It >> seems >> >> presumptuous call a module 'thing2' without the blessing of the >> 'thing' >> >> maintainers. They might want to use that name one day, like the >> Python >> >> 2.7+ >> >> unittest package is backported as unittest2. >> >> >> >> 'multiprocess' sounds liable to cause confusion - if I saw that and >> >> 'multiprocessing' in a tab completion list, I'd probably have to >> check >> >> which one was which. >> >> >> >> Thomas >> >> >> >> _______________________________________________ >> >> IPython-dev mailing list >> >> IPython-dev at scipy.org >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> >> >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> >> >> --- >> >> Mike McKerns >> http://www.its.caltech.edu/~mmckerns >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > ? > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > --- Mike McKerns http://www.its.caltech.edu/~mmckerns From mmckerns at caltech.edu Mon Jun 22 17:23:40 2015 From: mmckerns at caltech.edu (Michael McKerns) Date: Mon, 22 Jun 2015 17:23:40 -0400 Subject: [IPython-dev] drop-in replacement for multiprocessing Message-ID: <648c5c4b79b32a58073d837d4fac03e9.squirrel@webmail.caltech.edu> > > Yeah? a monkeypach is probably the right way to go... > I remember now why I didn't monkeypatch. A basic monkeypatch will replace the module in sys.modules after calling "mymodule.patch()"? but that only works for the python instance in the master? not on the workers. The workers would not have the patch applied unless you do some magic to bootstrap a "patch" call on any new process instantiation. So it suddenly becomes much harder to do the right thing. So maybe really the easier approach is submit a patch, as Thomas suggested. Oh, and, "pip install multiprocess" does work as of right now. I agree that it probably shouldn't, but it does. Min, I guess your patch to block that should be motivation enough for me to submit a patch to CPython to allow overriding the serializer in multiprocessing. --- Mike McKerns http://www.its.caltech.edu/~mmckerns From takowl at gmail.com Mon Jun 22 17:34:52 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 22 Jun 2015 14:34:52 -0700 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: <648c5c4b79b32a58073d837d4fac03e9.squirrel@webmail.caltech.edu> References: <648c5c4b79b32a58073d837d4fac03e9.squirrel@webmail.caltech.edu> Message-ID: On 22 June 2015 at 14:23, Michael McKerns wrote: > Oh, and, "pip install multiprocess" does work as > of right now. I agree that it probably shouldn't, > but it does. > I wouldn't be surprised if there was some combination of platform, Python installation, pip version, environment system and presence/absence of .pth files where it doesn't work even at present. Lots of stuff can mess with sys.path. ;-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Jun 22 17:48:49 2015 From: benjaminrk at gmail.com (MinRK) Date: Mon, 22 Jun 2015 14:48:49 -0700 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: <648c5c4b79b32a58073d837d4fac03e9.squirrel@webmail.caltech.edu> References: <648c5c4b79b32a58073d837d4fac03e9.squirrel@webmail.caltech.edu> Message-ID: On Mon, Jun 22, 2015 at 2:23 PM, Michael McKerns wrote: > > > > Yeah? a monkeypach is probably the right way to go... > > > > I remember now why I didn't monkeypatch. > > A basic monkeypatch will replace the module in > sys.modules after calling "mymodule.patch()"? > but that only works for the python instance in > the master? not on the workers. The workers > would not have the patch applied unless you > do some magic to bootstrap a "patch" call on > any new process instantiation. So it suddenly > becomes much harder to do the right thing. > > So maybe really the easier approach is submit > a patch, as Thomas suggested. > > Oh, and, "pip install multiprocess" does work as > of right now. Not in any of my envs (conda, virtualenv, Python 2.7/3.4, pip 7), it doesn't. It installs just fine, but is never imported because site-packages is lower priority than stdlib. -MinRK > I agree that it probably shouldn't, > but it does. Min, I guess your patch to block that > should be motivation enough for me to submit a > patch to CPython to allow overriding the serializer > in multiprocessing. > > --- > > Mike McKerns > http://www.its.caltech.edu/~mmckerns > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmckerns at caltech.edu Mon Jun 22 18:23:09 2015 From: mmckerns at caltech.edu (Michael McKerns) Date: Mon, 22 Jun 2015 18:23:09 -0400 Subject: [IPython-dev] drop-in replacement for multiprocessing In-Reply-To: References: <648c5c4b79b32a58073d837d4fac03e9.squirrel@webmail.caltech.edu> Message-ID: <1152e4ff2a48236a6bdc524009d4384b.squirrel@webmail.caltech.edu> Yeah, of course, that would do it. I meant to point out that it *installs* just fine with `pip`. I guess I'm being a dinosaur to still have site-packages before lib. Guys, thanks. This gives me a bunch to work with. > Not in any of my envs (conda, virtualenv, Python 2.7/3.4, pip 7), it > doesn't. It installs just fine, but is never imported because > site-packages is lower priority than stdlib. --- Mike McKerns http://www.its.caltech.edu/~mmckerns From 36rahu at gmail.com Tue Jun 23 02:59:41 2015 From: 36rahu at gmail.com (Rahul K P) Date: Tue, 23 Jun 2015 12:29:41 +0530 Subject: [IPython-dev] Flask app running interactively on IPython In-Reply-To: References: <16A43FEA-D2DC-4D77-AB38-DB15738885F8@gmail.com> Message-ID: I will describe with examples. TkinterCode.py ~~~~~~~~~ from Tkinter import * root = Tk() w = Label(root, text="Hello, world!") w.pack() try: print '__IPYTHON__ = ', __IPYTHON__ except: root.mainloop() Run this code on IPython prompt. Like run TkinterCode.py. After running we will get the command prompt. FLaskCode.py ~~~~~~~~~~~ from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello World!' if __name__ == '__main__': app.run() Run this code on IPython promt. Like run FlaskCode.py .After running we didn't get the command prompt. Is there any option to get the command prompt while running ? >> On Mon, Jun 22, 2015 at 11:06 PM, Thomas Kluyver wrote: > I assume you mean event loop integration. That's generally only done for > GUI event loops, but there's no reason in principle you couldn't integrate > with a web app event loop. > > See the docs here for more info: > http://ipython.org/ipython-doc/3/config/eventloops.html > > (I love having docs for stuff like this - it saves so much explanation > when someone asks about it) > > Thomas > > On 22 June 2015 at 04:58, Matthias Bussonnier < > bussonniermatthias at gmail.com> wrote: > >> Hi Rahul, >> >> > On Jun 22, 2015, at 09:27, Rahul K P <36rahu at gmail.com> wrote: >> > >> > Hi all, >> > Right now am working in image processing in Python. The interface >> has been build on TKinter at that time we can work the software >> interactively. The software run using the command run -i program.py. Right >> now we changed into flask interface and the IPython interactive coding not >> working here. We didn't get the command line while running. Anyone please >> help to solve this. >> >> We will need a bit more information or code example of what you are doing >> to be able to help. >> It would be nice to also understand what you are trying to achieve. >> >> Thanks, >> -- >> Matthias >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Jun 24 14:17:29 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 24 Jun 2015 11:17:29 -0700 Subject: [IPython-dev] Flask app running interactively on IPython In-Reply-To: References: <16A43FEA-D2DC-4D77-AB38-DB15738885F8@gmail.com> Message-ID: On 22 June 2015 at 23:59, Rahul K P <36rahu at gmail.com> wrote: > Is there any option to get the command prompt while running ? Yes, this is event loop integration. See the docs here for details on how to do it: http://ipython.org/ipython-doc/3/config/eventloops.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Jun 24 19:10:56 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 24 Jun 2015 16:10:56 -0700 Subject: [IPython-dev] Fwd: [julia-dev] Re: Machine learning In-Reply-To: <88854eee-77f4-4c8b-98c1-c946a41789e4@googlegroups.com> References: <5ad1a480-9e61-4306-ae1f-014b9182e735@googlegroups.com> <13d7c4a9-e113-4d49-966a-d6f10324d46d@googlegroups.com> <605a26ca-8c39-4314-8776-6f8d5abfb0f9@googlegroups.com> <5cf3c849-975f-489d-9ed5-6aad7e73b2f4@googlegroups.com> <51f069d5-67ca-4445-8433-f286e2a6a708@googlegroups.com> <10338568-80be-4f6d-936e-cc2544a25903@googlegroups.com> <37e6ef62-5483-45d4-8a27-fdf9caab699e@googlegroups.com> <88854eee-77f4-4c8b-98c1-c946a41789e4@googlegroups.com> Message-ID: Hi folks I know we have a number of devs on this list interested in improving the integration between IPython/Jupyter and PySpark, and today I found on a thread on the Julia-dev mailing list this particularly clear summary of how PySpark executes code and interacts with the JVM at a low level. I'd heard the story a couple of times, but had never seen it so concisely summarized with pointers to the specific locations in the code where the process boundaries are crossed. Very useful. Cheers f ---------- Forwarded message ---------- From: Andrei Zhabinski Date: Sat, Jun 20, 2015 at 9:22 AM Subject: Re: [julia-dev] Re: Machine learning To: julia-dev at googlegroups.com I feel like it's worth to describe how PySpark is implemented and what is needed to connect Julia to Spark in the same manner. In Spark, central concept is RDD - distributed collection of data partitions (splits). There are many different types of RDDs, such as MapPartitionsRDD, ShuffledRDD, CheckpointRDD, etc. Each type of RDD introduces conceptually new feature, e.g. MapPartitionsRDD is used to implement `map()`, `flatMap()`, `mapPartitions()` and similar methods, ShuffleRDD is responsible for shuffling data between machines, etc. To create new feature, every type of RDD should implement at least one method - `compute(split: Partition, context: TaskContext): Iterator[T]`. Essentially, `compute()` takes input partition data iterator and returns output data iterator. This is very similar to how `mapPartitions()` works - it simply applies some arbitrary transformation to every partition (and this is essentially what MapPartitionsRDD's `compute()` method does[1]). Some RDDs also involve data shuffling and working with external resources, but they aren't that important for our talk. PythonRDD comes in here. In its `compute()` method PythonRDD: 1) creates or reuses Python process [2] 2) writes serialized command and input data to the Python process (in a separate thread) [3] 3) reads results from the Python process [4] So Scala talks to Python process via socket interface using simple custom protocol. But essentially we want Python to talk to JVM and not vice versa. This is where Py4J is useful. PySpark driver creates JVM and uses it to maintain all needed objects (mainly, SparkContext and RDD). Python's RDD (i.e. "class RDD" in "rdd.py") keeps reference to the corresponding JVM object ("PythonRDD" in "PythonRDD.scala") and calls its methods. When we write something like this in PySpark: rdd = sc.textFile(...) rdd.map(lambda x: x**2) .collect() what happens is actually this: 1) Python's RDD is created pointing to PythonRDD object in JVM 2) subclass of Python's RDD - PipelinedRDD - is created; PipelinedRDD keeps reference to previous RDD and function `f = lambda x: x**2` to be applied to each record in original RDD 3) `PipelinedRDD.count()` leads to calling corresponding method in PythonRDD, then to passing all data through sockets to Python processes on workers, collecting results and passing it back to Python process on the driver machine. PipelinedRDD is called so because it can pipeline Julia functions in `map()` and `reduce()` operations without the need to go back to JVM, but this is mostly optimization. Some points are simplified or may contain errors, but essentially this is more or less how it works. So what do we need to implement Julia-Spark connector? Essentially, there are only 2 things - Julia-aware workers (JuliaRDD) and driver types/functions (RDD, PipelinedRDD) to call it directly from Julia. Not that much! [1]: https://github.com/apache/spark/blob/branch-1.4/core/src/main/scala/org/apache/spark/rdd/MapPartitionsRDD.scala#L34 [2]: https://github.com/apache/spark/blob/branch-1.4/core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala#L73 [3]: https://github.com/apache/spark/blob/branch-1.4/core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala#L208 [4]: https://github.com/apache/spark/blob/branch-1.4/core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala#L106 On Friday, June 19, 2015 at 11:16:20 PM UTC+3, Jeff Waller wrote: > > > > On Friday, June 19, 2015 at 1:26:55 PM UTC-4, wil... at gmail.com wrote: >> >> For a Spark integration is required a Java-Julia (de)serializer like it's >> done for Python and R. Python frontend based on Py4j. Similar RPC is done >> for Spark-R. Because Spark is bunch of transformation (read functions), it >> is needed to map these transformations into frontend language function >> calls. IMHO, Spark-Julia binding deserve a try, but I'm more inclined to >> pure Julia implementation of Spark transformations. Native Julia Spark >> package on top Elly.jl would be beat any JVM based implementation by >> performance and resource usage. >> > > I believe that getting something that actually runs will inspire people to > try it and make it better, and it (part 1) can be completed > before the end of the summer. > > Let me see if I can understand this problem. Are you saying it's this for > example > > PySpark > > someRDD.reduceByKey(lambda x,y: x+y) ---> mapped to Java via Py4j > > JuliaSpark > > reduceByKey(someRDD,(x,y)->x+y) ----> mapped to Java via X <--- what > does this need to be > > is the tricky part is coming up with a reasonable X? > > > -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From 36rahu at gmail.com Thu Jun 25 03:51:18 2015 From: 36rahu at gmail.com (Rahul K P) Date: Thu, 25 Jun 2015 13:21:18 +0530 Subject: [IPython-dev] Flask app running interactively on IPython In-Reply-To: References: <16A43FEA-D2DC-4D77-AB38-DB15738885F8@gmail.com> Message-ID: Thank you Thomas Kluyver, I got the idea about the event loop integrations. TKinter,PyQt4,Wx,Gtk,etc has event loop integrated with IPython. How implement event loop in flask ? On Wed, Jun 24, 2015 at 11:47 PM, Thomas Kluyver wrote: > On 22 June 2015 at 23:59, Rahul K P <36rahu at gmail.com> wrote: > >> Is there any option to get the command prompt while running ? > > > Yes, this is event loop integration. See the docs here for details on how > to do it: http://ipython.org/ipython-doc/3/config/eventloops.html > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Regards Rahul K P Research Engineer CastaliaLabs Pune +919895980223 -------------- next part -------------- An HTML attachment was scrubbed... URL: From claresloggett at gmail.com Thu Jun 25 10:36:12 2015 From: claresloggett at gmail.com (Clare Sloggett) Date: Fri, 26 Jun 2015 00:36:12 +1000 Subject: [IPython-dev] Internal port error from Jupyterhub In-Reply-To: References: Message-ID: Thanks MinRK and Thomas! Sorry not to think of something so simple. On 20 June 2015 at 03:54, MinRK wrote: > On Thu, Jun 18, 2015 at 11:57 PM, Clare Sloggett > wrote: > > Hi all, >> >> Is this mailing list still the right place to ask about Jupyterhub? >> > This is fine, but we?re gently directing Jupyter stuff toward the jupyter > mailing list . > > I'm getting an error that seems strange to me, I'm hoping someone might >> have insight. It's complaining about port availability, and it really looks >> like that's the error, but the port looks available. So I'm mystified. >> >> Short version: when I run >> jupyterhub --port 9520 --JupyterHub.hub_port=8500 >> I get an error ("Failed to bind hub to http://localhost:8500/hub/" and >> "Cannot assign requested address" - detailed error message is below). >> >> Yet, if I use the same port to run >> python -m SimpleHTTPServer 8500 >> the SimpleHTTPServer runs happily and is accessible, indicating that the >> port is actually available. >> >> This occurs no matter what port I specify for JupyterHub.hub_port. >> >> Here are some more details: >> >> I'm trying to install on an Ubuntu 14.04 server. There are other services >> running on this server, but none of them occupy port 8500 (or 9520) and >> given that SimpleHTTPServer works I am assuming that the other services are >> probably not the problem? >> > It?s possible that you can?t bind to localhost, not the port. > SimpleHTTPServer binds to all interfaces by default. Try: > > import socket > s = socket.socket() > s.bind(('localhost', 8500)) > > If that fails, you can try 127.0.0.1 instead of localhost. If 127 works, > you can set this value in your jupyterhub_config.py file: > > c.JupyterHub.hub_ip = '127.0.0.1' > > -MinRK > > >> To install Jupyterhub I am running >> >> git clone https://github.com/jupyter/jupyterhub >> >> cd jupyterhub/ >> >> sudo npm install -g configurable-http-proxy >> >> sudo apt-get install python3-pip >> >> sudo apt-get install nodejs-legacy >> >> sudo pip3 install -r requirements.txt >> sudo pip3 install . >> >> If I run Jupyterhub as root (or otherwise), the complete error I get is: >> >> $ sudo jupyterhub --port 9520 --JupyterHub.hub_port=8500 >> [I 2015-06-19 06:51:17.962 JupyterHub app:522] Loading cookie_secret from >> /home/ubuntu/install_jupyterhub/jupyterhub/jupyterhub_cookie_secret >> [W 2015-06-19 06:51:18.010 JupyterHub app:260] >> Generating CONFIGPROXY_AUTH_TOKEN. Restarting the Hub will require >> restarting the proxy. >> Set CONFIGPROXY_AUTH_TOKEN env or JupyterHub.proxy_auth_token config >> to avoid this message. >> >> [I 2015-06-19 06:51:18.020 JupyterHub app:623] Not using whitelist. Any >> authenticated user will be allowed. >> [E 2015-06-19 06:51:18.034 JupyterHub app:998] Failed to bind hub to >> http://localhost:8500/hub/ >> [E 2015-06-19 06:51:18.034 JupyterHub app:1066] >> Traceback (most recent call last): >> File "/usr/local/lib/python3.4/dist-packages/jupyterhub/app.py", >> line 1064, in launch_instance_async >> yield self.start() >> File "/usr/local/lib/python3.4/dist-packages/jupyterhub/app.py", >> line 996, in start >> self.http_server.listen(self.hub_port, address=self.hub_ip) >> File "/usr/local/lib/python3.4/dist-packages/tornado/tcpserver.py", >> line 126, in listen >> sockets = bind_sockets(port, address=address) >> File "/usr/local/lib/python3.4/dist-packages/tornado/netutil.py", >> line 187, in bind_sockets >> sock.bind(sockaddr) >> OSError: [Errno 99] Cannot assign requested address >> >> Any help would be very much appreciated! >> >> Cheers, >> Clare >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> ? > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.suchaneck at gmail.com Fri Jun 26 12:26:21 2015 From: a.suchaneck at gmail.com (Anton Suchaneck) Date: Fri, 26 Jun 2015 18:26:21 +0200 Subject: [IPython-dev] Single click copy+paste in IPython notebook? Message-ID: Hello! I'd like to use single click copy+paste in the IPython notebook - as it is found in some advanced editors. Single click means: * My cursor is at the destination position * I press the middle mouse and start selecting text at some other place. Meanwhile the actual cursor isn't moving * as soon as I release the middle mouse now, the selected text is inserted at the position where my curser was at the beginning. I used only a single mouse click and no keyboard for this. Is it possible to include this into the IPython notebook? Where would that be (extension, codemirror, browser, OS, ...)? Cheers, Anton -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Fri Jun 26 12:55:32 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 26 Jun 2015 09:55:32 -0700 Subject: [IPython-dev] Single click copy+paste in IPython notebook? In-Reply-To: References: Message-ID: You could have a go at doing that in an extension. It's probably too fragile to include in the notebook by default, given all the other things that middle click can mean. Thomas On 26 June 2015 at 09:26, Anton Suchaneck wrote: > Hello! > > I'd like to use single click copy+paste in the IPython notebook - as it is > found in some advanced editors. > Single click means: > * My cursor is at the destination position > * I press the middle mouse and start selecting text at some other place. > Meanwhile the actual cursor isn't moving > * as soon as I release the middle mouse now, the selected text is inserted > at the position where my curser was at the beginning. > I used only a single mouse click and no keyboard for this. > > Is it possible to include this into the IPython notebook? > Where would that be (extension, codemirror, browser, OS, ...)? > > Cheers, > Anton > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.suchaneck at gmail.com Fri Jun 26 12:58:33 2015 From: a.suchaneck at gmail.com (Anton Suchaneck) Date: Fri, 26 Jun 2015 18:58:33 +0200 Subject: [IPython-dev] Single click copy+paste in IPython notebook? In-Reply-To: References: Message-ID: So it would be possible? I currently only know Python and have no idea about internals of the notebook, Javascript or so. Could you give me a pointer what I need to learn and where I need to hook in to make this feature? 2015-06-26 18:55 GMT+02:00 Thomas Kluyver : > You could have a go at doing that in an extension. It's probably too > fragile to include in the notebook by default, given all the other things > that middle click can mean. > > Thomas > > On 26 June 2015 at 09:26, Anton Suchaneck wrote: > >> Hello! >> >> I'd like to use single click copy+paste in the IPython notebook - as it >> is found in some advanced editors. >> Single click means: >> * My cursor is at the destination position >> * I press the middle mouse and start selecting text at some other place. >> Meanwhile the actual cursor isn't moving >> * as soon as I release the middle mouse now, the selected text is >> inserted at the position where my curser was at the beginning. >> I used only a single mouse click and no keyboard for this. >> >> Is it possible to include this into the IPython notebook? >> Where would that be (extension, codemirror, browser, OS, ...)? >> >> Cheers, >> Anton >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Fri Jun 26 14:20:24 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Fri, 26 Jun 2015 20:20:24 +0200 Subject: [IPython-dev] Single click copy+paste in IPython notebook? In-Reply-To: References: Message-ID: Are you at scipy? Otherwise I think our talk on extensions Will be recorded, and we'll put the material online. Sent from my iPhone > On 26 juin 2015, at 18:58, Anton Suchaneck wrote: > > So it would be possible? > I currently only know Python and have no idea about internals of the notebook, Javascript or so. > Could you give me a pointer what I need to learn and where I need to hook in to make this feature? > > 2015-06-26 18:55 GMT+02:00 Thomas Kluyver : >> You could have a go at doing that in an extension. It's probably too fragile to include in the notebook by default, given all the other things that middle click can mean. >> >> Thomas >> >>> On 26 June 2015 at 09:26, Anton Suchaneck wrote: >>> Hello! >>> >>> I'd like to use single click copy+paste in the IPython notebook - as it is found in some advanced editors. >>> Single click means: >>> * My cursor is at the destination position >>> * I press the middle mouse and start selecting text at some other place. Meanwhile the actual cursor isn't moving >>> * as soon as I release the middle mouse now, the selected text is inserted at the position where my curser was at the beginning. >>> I used only a single mouse click and no keyboard for this. >>> >>> Is it possible to include this into the IPython notebook? >>> Where would that be (extension, codemirror, browser, OS, ...)? >>> >>> Cheers, >>> Anton >>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From damienlmoore at gmail.com Fri Jun 26 17:16:57 2015 From: damienlmoore at gmail.com (Damien Moore) Date: Fri, 26 Jun 2015 17:16:57 -0400 Subject: [IPython-dev] More reactive autocompletion in the notebook Message-ID: Hi, I'm new user to IPython and like what I see so far. I'm glad to see that there's a recent push to improve the getting started documentation and the new user experience. It took me a little searching to figure out how it all works, from starting the server, and getting the notebook to open in the correct browser. For someone new to python altogether, this would be a huge turn off. (Just understanding that ipython/jupyter has a client-server architecture will be a big leap for a newbie) Onto my actual question: I am used to using IDEs with more reactve autocompletion. Generally if you type 2-3 characters, you will get completion hints popup automatically. Typing a period will popup the list of module or class members. Type an opening brace on a function and you see the callsig, docstring etc for the function. Currently the user manually needs to type tab or shift-tab to get this info -- which is clunky and also less discoverable. Is there a way to configure the notebook to automatically pop up these hints, or if not, is there any interest in implementing such a feature? Thanks, Damien -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.suchaneck at gmail.com Fri Jun 26 17:40:39 2015 From: a.suchaneck at gmail.com (Anton Suchaneck) Date: Fri, 26 Jun 2015 23:40:39 +0200 Subject: [IPython-dev] Fully customize IPython notebook highlighting? Message-ID: Hey! I've attempted to customize the syntax highlighting for the IPython notebook with custom.css I couldn't set a color for brackets through cm-bracket. Is it not connected? Also I couldn't find any useful effect from cm-atom, cm-property, cm-attribute, cm-variable-2. Are these classes not functional and where can I find more information about it? What is the simplest way to include some custom highlighting? For example color everything before a "=" or even a more complex regex. Cheers, Anton -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Fri Jun 26 17:56:20 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Fri, 26 Jun 2015 23:56:20 +0200 Subject: [IPython-dev] Fully customize IPython notebook highlighting? In-Reply-To: References: Message-ID: <2629E105-19C6-40AF-AB2C-8A113357DB68@gmail.com> Hi Anton, > On Jun 26, 2015, at 23:40, Anton Suchaneck wrote: > > I've attempted to customize the syntax highlighting for the IPython notebook with custom.css > I couldn't set a color for brackets through cm-bracket. Is it not connected? > Also I couldn't find any useful effect from cm-atom, cm-property, cm-attribute, cm-variable-2. > > Are these classes not functional and where can I find more information about it? All all the highlighting is done by code mirror, nothing is IPython specific. > > What is the simplest way to include some custom highlighting? For example color everything before a "=" or even a more complex regex. I suggest you have a look at CodeMirror Modes Api http://codemirror.net/doc/manual.html#modeapi This explain to you what a mode is, and how it works. Codemirror Mailing List / forum will be more appropriate. Once you know how to do what you want in CodeMirror we can help you to apply it to IPython. Cheers, -- M -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Fri Jun 26 19:09:50 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 26 Jun 2015 16:09:50 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 Message-ID: I'd like to float the idea of a soft moratorium on big architectural changes in the notebook between 4.0 and 5.0. That is, as 4.0 will mostly be 3.x's features on a new foundation, 5.0 would be mostly the 4.0's foundations with added & polished features. With 4.0, we'll have made two consecutive major releases where the headline changes are architectural: the kernelspec machinery and changes for language agnosticity in 3.0, and the Big Split in 4.0. We've done all of this for good reasons, but I worry that we're becoming captivated by the endless programmer dream of creating the perfect framework to solve all our problems. One of the principles to which we attribute the success of the project so far is that we solve the problems in front of us and restructure when we need to, rather than rushing into building grand architecture. I think that we need a cycle to digest the architectural changes we've recently made before we dive into another round. In particular, the talk of refactoring all our frontend machinery concerns me. I don't want to stop us from planning this, but I think it would be a mistake to try to do it for 5.0. There are a lot of important features that we have been saying for some time we will work on - like multi cell selection, find & replace across the notebook, improved tab completion, and an HTML console interface. We've deferred these tasks while we worked on the architecture, and it's starting to be embarrassing that the interface is not moving forwards quicker. There would also be less impetus for people to turn to competing projects if we sanded some of the rough edges off the notebook interface. On that note, I'd like us to try doing 'user Fridays', where we pretend that we can't change the Jupyter/IPython code, and work on making interesting notebooks, extensions, and other parts of the ecosystem. Every time I work on such things, it highlights problems I wasn't aware of, or rough edges I hadn't fully appreciated. Having a semi-official mechanism to encourage us to do this in regular small chunks would be valuable in keeping us focussed on users rather than just architecture. The moratorium I propose would be left to our judgement to implement - it's a principle rather than a rule. I'd make an exception for the conversion to Typescript, because Matthias has already spent significant time on that, and I know he's champing at the bit to finish it. Widgets would be exempt, because we've made it very clear that they're liable to change significantly, and they shouldn't much affect development of the notebook itself. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Jun 26 19:45:43 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 26 Jun 2015 16:45:43 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: On Fri, Jun 26, 2015 at 4:09 PM, Thomas Kluyver wrote: > The moratorium I propose would be left to our judgement to implement - > it's a principle rather than a rule. I'd make an exception for the > conversion to Typescript, because Matthias has already spent significant > time on that, and I know he's champing at the bit to finish it. Widgets > would be exempt, because we've made it very clear that they're liable to > change significantly, and they shouldn't much affect development of the > notebook itself. While I hear very much the spirit of what you are saying, and I certainly think that we can't lose sight that the *only* thing that ultimately matters is whether we serve our users well or not, there's a big piece that is already burning under us that probably can't wait. In fact, at the last dev meeting, Jason already posted his new draft code in this direction: https://github.com/jasongrout/phosphor-notebook and this is part of the reason why Matthias has been working on the Typescript conversion, and also one of the big bottleneck points: we know that multiple folks want to customize the notebook UI more heavily, and in its current incarnation, that's very, very difficult. And for multiple teams working in different directions, probably impractical/impossible at the end of the day. The current Notebook is, in a certain sense, where the core IPython code was sometime in the 0.9-0.10 period: highly functional, yet highly monolithic and very difficult to change, adapt or extend. And I really think that, just as much as eventually we had to bite that bullet, and how we benefitted from doing so at the time, we have to do the UI restructuring now. I really don't see how, right now, multiple folks are going to be able to simultaneously work on the notebook codebase being as monolithic as it is. So yes, I do hear you, and I think it's important that we don't lose sight of the value for our users. We can't become a project that only looks at its own development needs and never produces features that actually matter to its users. That's more or less what Python did with Python 3, and we all know how that went... But I do think that we've accrued sufficient technical debt with the codebase of the notebook, and we have enough pressure *already at our doorstep* from multiple fronts regarding the need to implement many different types of tools and ideas atop the web frontend, that we need to open that up. What I hope is that over the next few days at Scipy, when most of you (sadly without me) have a chance to meet face to face, you'll be able to hatch a plan to do this in a way that doesn't lose track of the important goals: - do a minimal refactoring that preserves today's functionality, even if there's perhaps a small loss in polish on first release. - get a more decoupled codebase that will allow us to build the various pieces we need. But I honestly don't see the current notebook codebase supporting the pressure we have already on it without a refactoring right away, precisely so that we can let more folks build new functionality... Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From sccolbert at gmail.com Fri Jun 26 20:38:12 2015 From: sccolbert at gmail.com (Chris Colbert) Date: Fri, 26 Jun 2015 20:38:12 -0400 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: I have to side firmly with Fernando on this one. After ripping apart the notebook code base over the last few days, it's become clear that things need change measurably if we are going to move forward with a platform that is both maintainable and customizable. There is quite a bit of technical debt in there, and that bill is coming due. My team and cohorts are ready, willing, and able to tackle this problem (as Fernando mentioned, we've already started), and I think it would be a strategic mistake to shut the door on the momentum we've been building over the last several months. On Fri, Jun 26, 2015 at 7:45 PM, Fernando Perez wrote: > > On Fri, Jun 26, 2015 at 4:09 PM, Thomas Kluyver wrote: > >> The moratorium I propose would be left to our judgement to implement - >> it's a principle rather than a rule. I'd make an exception for the >> conversion to Typescript, because Matthias has already spent significant >> time on that, and I know he's champing at the bit to finish it. Widgets >> would be exempt, because we've made it very clear that they're liable to >> change significantly, and they shouldn't much affect development of the >> notebook itself. > > > While I hear very much the spirit of what you are saying, and I certainly > think that we can't lose sight that the *only* thing that ultimately > matters is whether we serve our users well or not, there's a big piece that > is already burning under us that probably can't wait. In fact, at the last > dev meeting, Jason already posted his new draft code in this direction: > > https://github.com/jasongrout/phosphor-notebook > > and this is part of the reason why Matthias has been working on the > Typescript conversion, and also one of the big bottleneck points: we know > that multiple folks want to customize the notebook UI more heavily, and in > its current incarnation, that's very, very difficult. And for multiple > teams working in different directions, probably impractical/impossible at > the end of the day. > > The current Notebook is, in a certain sense, where the core IPython code > was sometime in the 0.9-0.10 period: highly functional, yet highly > monolithic and very difficult to change, adapt or extend. And I really > think that, just as much as eventually we had to bite that bullet, and how > we benefitted from doing so at the time, we have to do the UI restructuring > now. > > I really don't see how, right now, multiple folks are going to be able to > simultaneously work on the notebook codebase being as monolithic as it is. > > So yes, I do hear you, and I think it's important that we don't lose sight > of the value for our users. We can't become a project that only looks at > its own development needs and never produces features that actually matter > to its users. That's more or less what Python did with Python 3, and we > all know how that went... > > But I do think that we've accrued sufficient technical debt with the > codebase of the notebook, and we have enough pressure *already at our > doorstep* from multiple fronts regarding the need to implement many > different types of tools and ideas atop the web frontend, that we need to > open that up. > > What I hope is that over the next few days at Scipy, when most of you > (sadly without me) have a chance to meet face to face, you'll be able to > hatch a plan to do this in a way that doesn't lose track of the important > goals: > > - do a minimal refactoring that preserves today's functionality, even if > there's perhaps a small loss in polish on first release. > > - get a more decoupled codebase that will allow us to build the various > pieces we need. > > But I honestly don't see the current notebook codebase supporting the > pressure we have already on it without a refactoring right away, precisely > so that we can let more folks build new functionality... > > Cheers, > > f > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Jun 26 20:57:14 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 26 Jun 2015 17:57:14 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: On Fri, Jun 26, 2015 at 5:38 PM, Chris Colbert wrote: > I have to side firmly with Fernando on this one. After ripping apart the > notebook code base over the last few days, it's become clear that things > need change measurably if we are going to move forward with a platform that > is both maintainable and customizable. There is quite a bit of technical > debt in there, and that bill is coming due. My team and cohorts are ready, > willing, and able to tackle this problem (as Fernando mentioned, we've > already started), and I think it would be a strategic mistake to shut the > door on the momentum we've been building over the last several months. > What's going to be very challenging in this effort is the *coordination* part. We need to make sure that we have both a chance for proper discussion of the key ideas, and for open coordination of the effort while it's happening, so that: a) whoever ends up working on the critical path can focus on that and just get the nasty business done b) the rest of the team can feel comfortable that there's still other areas where work can proceed cleanly and without interference or blockage. In the past we've been reasonably good at these kinds of things, but the project is getting bigger, more complicated, and we have more people too. So the challenge this time is going to be significantly harder. Let's put in the necessary effort also into communicating how to do this right, so we come out ahead with a cleaner codebase we're all happier working on in short order... It would be great if during scipy, you folks try to draft an outline of what the key pieces of that refactoring would entail, identifying what areas of the project would effectively hold locks. That would let us more easily define what is NOT locked, so we can then partition things for everyone else to work on. Let's have that publicly documented (probably as an IPEP) so that we don't break the team in the process... Cheers f -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Sat Jun 27 02:25:24 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 26 Jun 2015 23:25:24 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: > One of the principles to which we attribute the success of the project so > far is that we solve the problems in front of us and restructure when we > need to, rather than rushing into building grand architecture. I think that > we need a cycle to digest the architectural changes we've recently made > before we dive into another round. In particular, the talk of refactoring > all our frontend machinery concerns me. I don't want to stop us from > planning this, but I think it would be a mistake to try to do it for 5.0. I do think there are part of our code base where conservatism is useful and beneficial: the message specification, notebook document format, kernel spec, etc. Many features and APIs associated with the ipython kernel are also evolving in a pretty conservative manner. The server side of the notebook is reasonably stable, although the real-time collab and ongoing security efforts will require significant changes to parts of it. However, I think the project would be deeply hurt by applying this conservatism broadly or to the notebook frontend in particular. > > There are a lot of important features that we have been saying for some time > we will work on - like multi cell selection, find & replace across the > notebook, improved tab completion, and an HTML console interface. We've > deferred these tasks while we worked on the architecture, and it's starting > to be embarrassing that the interface is not moving forwards quicker. There > would also be less impetus for people to turn to competing projects if we > sanded some of the rough edges off the notebook interface. I agree that we have lots of rough edge that are adversely affecting our users today. These range from bugs, usability problems, basic features that are lacking, and new, highly-anticipated features. However, the development challenges we have seen in trying to implement these things strongly point to continuing with a deep architectural surgery: * Off the top of my head, between Cal Poly, Berkeley, Google, Bloomberg and Continuum I can count 8-ish existing staff being allocated to work full time on the notebook. With upcoming new hires at Cal Poly and Berkeley, this number will go over >10 staff working primarily on the notebook. * Our existing monolithic, tightly-coupled frontend code makes it impossible for more than 1-2 main lines of frontend work to move forward at the same time. * We have numerous third part projects and companies (kbase, Rodeo, Hydrogen, Sage, IBM, Quantopian, DataRobot, O'Reilly, etc.) who are taking our frontend code and using it to build customized frontend applications. Our code base is making this extremely difficult for these groups. Because of this, each of these groups is having to maintain and develop forks, rather than working with us to build a common set of flexible components. This means that everyone is resolving the same problems in ways that our users don't benefit from. Deep changes in the frontend code are needed, precisely because there is no way we can build the user-focused features with this many people using our currently code base. Even something like the HTML console interface will be dramatically easier - if not trivial - once the code base is refactored into better-designed, loosely coupled components. > On that note, I'd like us to try doing 'user Fridays', where we pretend that > we can't change the Jupyter/IPython code, and work on making interesting > notebooks, extensions, and other parts of the ecosystem. Every time I work > on such things, it highlights problems I wasn't aware of, or rough edges I > hadn't fully appreciated. Having a semi-official mechanism to encourage us > to do this in regular small chunks would be valuable in keeping us focussed > on users rather than just architecture. I agree this is useful for all of us to do on a continual basis. I just finished teaching for ten weeks and most of my time was spent in doing this type of stuff. It was wonderful and very instructive. > The moratorium I propose would be left to our judgement to implement - it's > a principle rather than a rule. I'd make an exception for the conversion to > Typescript, because Matthias has already spent significant time on that, and > I know he's champing at the bit to finish it. Widgets would be exempt, > because we've made it very clear that they're liable to change > significantly, and they shouldn't much affect development of the notebook > itself. > > Thanks, > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From ellisonbg at gmail.com Sat Jun 27 02:30:19 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 26 Jun 2015 23:30:19 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: > What's going to be very challenging in this effort is the *coordination* > part. We need to make sure that we have both a chance for proper discussion > of the key ideas, and for open coordination of the effort while it's > happening, so that: Yep, I definitely agree that the coordination, organizational and communication aspects of the project will have to evolve in parallel to the code base. > > a) whoever ends up working on the critical path can focus on that and just > get the nasty business done > > b) the rest of the team can feel comfortable that there's still other areas > where work can proceed cleanly and without interference or blockage. My hope is that with the right technical choices and coordination, that we can remove the critical paths altogether. I have ideas on how to pull this off, and will post separately to the list about that. > It would be great if during scipy, you folks try to draft an outline of what > the key pieces of that refactoring would entail, identifying what areas of > the project would effectively hold locks. That would let us more easily > define what is NOT locked, so we can then partition things for everyone else > to work on. Yes, I think that should be one of the main goals. But I want the discussion to be focused on *how* we do this, not *if*. > > Let's have that publicly documented (probably as an IPEP) so that we don't > break the team in the process... Yep! Cheers, Brian > > Cheers > > f > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From wes.turner at gmail.com Sat Jun 27 02:36:22 2015 From: wes.turner at gmail.com (Wes Turner) Date: Sat, 27 Jun 2015 01:36:22 -0500 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: On Jun 27, 2015 1:26 AM, "Brian Granger" wrote: > > > One of the principles to which we attribute the success of the project so > > far is that we solve the problems in front of us and restructure when we > > need to, rather than rushing into building grand architecture. I think that > > we need a cycle to digest the architectural changes we've recently made > > before we dive into another round. In particular, the talk of refactoring > > all our frontend machinery concerns me. I don't want to stop us from > > planning this, but I think it would be a mistake to try to do it for 5.0. > > I do think there are part of our code base where conservatism is > useful and beneficial: the message specification, notebook document > format, kernel spec, etc. Many features and APIs associated with the > ipython kernel are also evolving in a pretty conservative manner. The > server side of the notebook is reasonably stable, although the > real-time collab and ongoing security efforts will require significant > changes to parts of it. > > However, I think the project would be deeply hurt by applying this > conservatism broadly or to the notebook frontend in particular. > > > > > There are a lot of important features that we have been saying for some time > > we will work on - like multi cell selection, find & replace across the > > notebook, improved tab completion, and an HTML console interface. We've > > deferred these tasks while we worked on the architecture, and it's starting > > to be embarrassing that the interface is not moving forwards quicker. There > > would also be less impetus for people to turn to competing projects if we > > sanded some of the rough edges off the notebook interface. > > I agree that we have lots of rough edge that are adversely affecting > our users today. These range from bugs, usability problems, basic > features that are lacking, and new, highly-anticipated features. > However, the development challenges we have seen in trying to > implement these things strongly point to continuing with a deep > architectural surgery: > > * Off the top of my head, between Cal Poly, Berkeley, Google, > Bloomberg and Continuum I can count 8-ish existing staff being > allocated to work full time on the notebook. With upcoming new hires > at Cal Poly and Berkeley, this number will go over >10 staff working > primarily on the notebook. > * Our existing monolithic, tightly-coupled frontend code makes it > impossible for more than 1-2 main lines of frontend work to move > forward at the same time. > * We have numerous third part projects and companies (kbase, Rodeo, > Hydrogen, Sage, IBM, Quantopian, DataRobot, O'Reilly, etc.) who are > taking our frontend code and using it to build customized frontend > applications. Our code base is making this extremely difficult for > these groups. Because of this, each of these groups is having to > maintain and develop forks, rather than working with us to build a > common set of flexible components. This means that everyone is > resolving the same problems in ways that our users don't benefit from. > > Deep changes in the frontend code are needed, precisely because there > is no way we can build the user-focused features with this many people > using our currently code base. Even something like the HTML console > interface will be dramatically easier - if not trivial - once the code > base is refactored into better-designed, loosely coupled components. I'm impressed by the notablemind react components: https://github.com/notablemind/notablemind/tree/master/app/components These seem to solve for the non-ot.js real-time parts as well. (See: etherpad-lite, apache wave protocol). > > > On that note, I'd like us to try doing 'user Fridays', where we pretend that > > we can't change the Jupyter/IPython code, and work on making interesting > > notebooks, extensions, and other parts of the ecosystem. Every time I work > > on such things, it highlights problems I wasn't aware of, or rough edges I > > hadn't fully appreciated. Having a semi-official mechanism to encourage us > > to do this in regular small chunks would be valuable in keeping us focussed > > on users rather than just architecture. > > I agree this is useful for all of us to do on a continual basis. I > just finished teaching for ten weeks and most of my time was spent in > doing this type of stuff. It was wonderful and very instructive. > > > The moratorium I propose would be left to our judgement to implement - it's > > a principle rather than a rule. I'd make an exception for the conversion to > > Typescript, because Matthias has already spent significant time on that, and > > I know he's champing at the bit to finish it. Widgets would be exempt, > > because we've made it very clear that they're liable to change > > significantly, and they shouldn't much affect development of the notebook > > itself. > > > > Thanks, > > Thomas > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Sat Jun 27 03:12:16 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Sat, 27 Jun 2015 00:12:16 -0700 Subject: [IPython-dev] More reactive autocompletion in the notebook In-Reply-To: References: Message-ID: Will Whitney is going to be doing some work on our tab completion system this summer and we will hopefully make improvements along these lines. On Fri, Jun 26, 2015 at 2:16 PM, Damien Moore wrote: > Hi, > > I'm new user to IPython and like what I see so far. > > I'm glad to see that there's a recent push to improve the getting started > documentation and the new user experience. It took me a little searching to > figure out how it all works, from starting the server, and getting the > notebook to open in the correct browser. For someone new to python > altogether, this would be a huge turn off. (Just understanding that > ipython/jupyter has a client-server architecture will be a big leap for a > newbie) > > Onto my actual question: I am used to using IDEs with more reactve > autocompletion. Generally if you type 2-3 characters, you will get > completion hints popup automatically. Typing a period will popup the list of > module or class members. Type an opening brace on a function and you see the > callsig, docstring etc for the function. Currently the user manually needs > to type tab or shift-tab to get this info -- which is clunky and also less > discoverable. Is there a way to configure the notebook to automatically pop > up these hints, or if not, is there any interest in implementing such a > feature? > > Thanks, > Damien > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From bussonniermatthias at gmail.com Sat Jun 27 05:55:20 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Sat, 27 Jun 2015 11:55:20 +0200 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: Hi All, Thanks for doing this on the public Mailing list, it seem we are improving our communication skills, be careful though, the response start to have more and more implicit knowledge. I do hear what Thomas is saying too, but I have seen to numerous time people reimplementing in javascript pieces of our codes because they are un-reusable (hydrogen, notable mind, the things that render notebook purely in js ... ) so it seem that **some** refactoring is needed. I?m not really happy to have done the conversion to typescript without API changes, and not got my PRs merged, and it seemed like these last month any changes to the notebook has had a strong pushback. And without theses changes we basically won?t have any new features. That seem (to me) like a pretty stable javascript Api for something we said can change at any time as we **want** to refactor. Writing some of our SciPy tutorial also made me realize how some low level feature of the notebook API (I know, not stable) are broken in design, and hopefully not used to much. So I really hope we clean things. That being said, I would also really like to spend more time **using** the notebook, to do "user Friday? and show really cool things that can be done. Though I think it is something which is hard to do with the current technical dept. -- M > On Jun 27, 2015, at 08:30, Brian Granger wrote: > >> What's going to be very challenging in this effort is the *coordination* >> part. We need to make sure that we have both a chance for proper discussion >> of the key ideas, and for open coordination of the effort while it's >> happening, so that: > > Yep, I definitely agree that the coordination, organizational and > communication aspects of the project will have to evolve in parallel > to the code base. > >> >> a) whoever ends up working on the critical path can focus on that and just >> get the nasty business done >> >> b) the rest of the team can feel comfortable that there's still other areas >> where work can proceed cleanly and without interference or blockage. > > My hope is that with the right technical choices and coordination, > that we can remove the critical paths altogether. I have ideas on how > to pull this off, and will post separately to the list about that. > >> It would be great if during scipy, you folks try to draft an outline of what >> the key pieces of that refactoring would entail, identifying what areas of >> the project would effectively hold locks. That would let us more easily >> define what is NOT locked, so we can then partition things for everyone else >> to work on. > > Yes, I think that should be one of the main goals. But I want the > discussion to be focused on *how* we do this, not *if*. > >> >> Let's have that publicly documented (probably as an IPEP) so that we don't >> break the team in the process... > > Yep! > > Cheers, > > Brian > >> >> Cheers >> >> f >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From a.suchaneck at gmail.com Sat Jun 27 08:36:12 2015 From: a.suchaneck at gmail.com (Anton Suchaneck) Date: Sat, 27 Jun 2015 14:36:12 +0200 Subject: [IPython-dev] Debugging when communication of notebook with kernel lost? Message-ID: Hi! I occasionally have the issue that no more commands are being executed in the IPython notebook. The interface responds, but just leaves [*] cells without progressing or interrupting on request. Sometimes this even happens right after starting. I cannot find an error message. Often restarting the kernel works, but sometimes even that doesn't work right away. How can I inspect what's going on (and maybe report that)? I work on a local Windows 7 computer with company firewall. Someone suggested it's due to Windows. Is there a log to enable so that I can report that? It's really annoying when working with data and suddenly having all data in the memory unaccessible. Regards, Anton -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Sat Jun 27 09:31:54 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Sat, 27 Jun 2015 15:31:54 +0200 Subject: [IPython-dev] Debugging when communication of notebook with kernel lost? In-Reply-To: References: Message-ID: <93AA912F-85E7-4D39-8278-F9F0E8BE9C9D@gmail.com> Hi Anton, > On Jun 27, 2015, at 14:36, Anton Suchaneck wrote: > > Hi! > > I occasionally have the issue that no more commands are being executed in the IPython notebook. The interface responds, but just leaves [*] cells without progressing or interrupting on request. Sometimes this even happens right after starting. I cannot find an error message. Often restarting the kernel works, but sometimes even that doesn't work right away. This look really like 1) the kernel is actually doing computation 2) the packets get intercepted/dropped in between fronted/server. I doubt this is 1, so it is most likely 2, unfortunately if it is 2, there will probably not be errors, are we are just waiting for things that will never come back. > How can I inspect what's going on (and maybe report that)? I would suggest starting the notebook with --debug, to get more verbose messages if any, and can you look at the Javascript console to see if there is anything ? > I work on a local Windows 7 computer with company firewall. Someone suggested it's due to Windows. Do you have sophos antivirus ? Cheers, -- M From a.suchaneck at gmail.com Sat Jun 27 14:13:28 2015 From: a.suchaneck at gmail.com (Anton Suchaneck) Date: Sat, 27 Jun 2015 20:13:28 +0200 Subject: [IPython-dev] Single click copy+paste in IPython notebook? In-Reply-To: References: Message-ID: I'm not at scipy, but I try to keep up with videos. I'll be excited to watch the video. I'd also appreciate any pointers to documentation I might need for this particular task and the IPython hooks (classes, methods, ...) I need to accomplish a copy/paste extension. Or what do you think: Is it more likely or easier to introduce such a copy/paste functionality rather as a browser plugin or even globally in the OS (Windows or Linux)? Thanks Matthias! 2015-06-26 20:20 GMT+02:00 Matthias Bussonnier : > Are you at scipy? Otherwise I think our talk on extensions Will be > recorded, and we'll put the material online. > > Sent from my iPhone > > On 26 juin 2015, at 18:58, Anton Suchaneck wrote: > > So it would be possible? > I currently only know Python and have no idea about internals of the > notebook, Javascript or so. > Could you give me a pointer what I need to learn and where I need to hook > in to make this feature? > > 2015-06-26 18:55 GMT+02:00 Thomas Kluyver : > >> You could have a go at doing that in an extension. It's probably too >> fragile to include in the notebook by default, given all the other things >> that middle click can mean. >> >> Thomas >> >> On 26 June 2015 at 09:26, Anton Suchaneck wrote: >> >>> Hello! >>> >>> I'd like to use single click copy+paste in the IPython notebook - as it >>> is found in some advanced editors. >>> Single click means: >>> * My cursor is at the destination position >>> * I press the middle mouse and start selecting text at some other place. >>> Meanwhile the actual cursor isn't moving >>> * as soon as I release the middle mouse now, the selected text is >>> inserted at the position where my curser was at the beginning. >>> I used only a single mouse click and no keyboard for this. >>> >>> Is it possible to include this into the IPython notebook? >>> Where would that be (extension, codemirror, browser, OS, ...)? >>> >>> Cheers, >>> Anton >>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Mon Jun 29 00:03:21 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Sun, 28 Jun 2015 21:03:21 -0700 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: > **please** please, try to distinguish one way communication from discussion on the ML. > Announcement are hard to refer to to and google and the ML. Announcement are made for the blog. > If you don?t expect that response are key to communication, use the blog, > you can start a thread **About** a blog post, but [Ann] on ML hurt communication. I view the blog as being reserved for highly curated and copy-edited content. That content is very time consuming to write and I don't want to discourage smaller one-way announcements by telling people that they all need to be blog posts. Also the blog serves a much difference audience - users - than a development focused mailing list. For development purposes, I don't see a significant difference between announcements and 2 ways communications. Here is another metric to use in deciding what is blog post worthy - do we want to tweet about it. Most things on the mailing list - even when 1 way - don't pass that test. Cheers, Brian -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From ellisonbg at gmail.com Mon Jun 29 00:20:59 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Sun, 28 Jun 2015 21:20:59 -0700 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: <4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE@gmail.com> Message-ID: I propose we frame the reports in the form of the following 3 questions: * What have I been working on? * What do I plan on working on? * What things are preventing me from making progress? With these questions, the goals of the meeting would be: * Keep everyone working as effectively as possible. * Give everyone enough information about what others are working to know what conversations they need to have. Also, I am thinking about 5 minutes/person is probably about max. I think that longer reports that have broad appeal (conferences, collaboration meetings, long lines of technical work) should be made on the main mailing lists. Cheers, Brian On Sun, Jun 28, 2015 at 9:17 PM, Brian Granger wrote: >>> I think we should limit the person time though - maybe 7 minutes? >> >> I proposed 10 min 3 weeks ago. > > Yes and it was a good idea! The only problem is that 10 minutes is > incompatible with a meeting whole goal is to make a lot of technical > decisions. We found that out pretty quickly. > > Upon thinking more, I am thinking that putting a 5 minute max person > might even be too high. > >> >> I would also really appreciate to not have 3 words like: ?Tag?. An entry on >> the hackpad should be self explanatory (unless link to an issues which is self explanatory). >> Like if the person who wrote that is not here we can decide without them. > > But I think the main content of Fernando's proposal is that we won't > ever make decisions in the dev meetings. > >> >> Have 1 person responsible for taking notes, and turn the hangout into a written small report, >> that we (eventually) post on the blog to keep reader in the loop. > > I don't think the blog post is appropriate for most of this stuff - > and I don't think we have anyone that wants to take the time to turn > the summary in to a weekly blog post yet. > > >>> We will probably have to think and talk further about the best way of >>> gathering the right sets of people at the right times and propagating >>> the information out, but this is a good next step. >> >> You mean we still don?t know how to open an issue on GitHub ? > > Of course not. But as the project grows, people will be working in a > more focused and specialized manner, unaware of what is going on in > the rest of the project. Even today, with ~15-20 repos, not all of us > are subscribed to all of them. We need ways of propagating information > from all that development activity on all of those repos in a way that > project leadership is able to guide the whole project in a coherent > manner. Also, remember, we are hiring a number of non-coders who will > need to participate in the project. > >> Seem pretty obvious, open an issue, people can subscribe or unsubscribe >> on a per issue bassi and coordinate. > > I don't think that is reasonable. We have to start having people > focusing and specializing and part of that is that more and more > people will be involved in the project, but not coding. We will soon > have people focused on design, fundraising, event planning, > budget/admin, hiring, managing staff, industry relations, etc. We have > a huge challenge in integrating these efforts into the code-centric > core of the project, but we have to. > >> Even when you have your earplugs you also generate echo. >> :-) > > Yes, that one was super weird - have never seen it before or since. > > Cheers, > > Brian > >> >> -- >> M >> >> -- >> You received this message because you are subscribed to the Google Groups "Project Jupyter" group. >> To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscribe at googlegroups.com. >> To post to this group, send email to jupyter at googlegroups.com. >> To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE%40gmail.com. >> For more options, visit https://groups.google.com/d/optout. > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From ellisonbg at gmail.com Mon Jun 29 00:17:15 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Sun, 28 Jun 2015 21:17:15 -0700 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: <4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE@gmail.com> References: <4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE@gmail.com> Message-ID: >> I think we should limit the person time though - maybe 7 minutes? > > I proposed 10 min 3 weeks ago. Yes and it was a good idea! The only problem is that 10 minutes is incompatible with a meeting whole goal is to make a lot of technical decisions. We found that out pretty quickly. Upon thinking more, I am thinking that putting a 5 minute max person might even be too high. > > I would also really appreciate to not have 3 words like: ?Tag?. An entry on > the hackpad should be self explanatory (unless link to an issues which is self explanatory). > Like if the person who wrote that is not here we can decide without them. But I think the main content of Fernando's proposal is that we won't ever make decisions in the dev meetings. > > Have 1 person responsible for taking notes, and turn the hangout into a written small report, > that we (eventually) post on the blog to keep reader in the loop. I don't think the blog post is appropriate for most of this stuff - and I don't think we have anyone that wants to take the time to turn the summary in to a weekly blog post yet. >> We will probably have to think and talk further about the best way of >> gathering the right sets of people at the right times and propagating >> the information out, but this is a good next step. > > You mean we still don?t know how to open an issue on GitHub ? Of course not. But as the project grows, people will be working in a more focused and specialized manner, unaware of what is going on in the rest of the project. Even today, with ~15-20 repos, not all of us are subscribed to all of them. We need ways of propagating information from all that development activity on all of those repos in a way that project leadership is able to guide the whole project in a coherent manner. Also, remember, we are hiring a number of non-coders who will need to participate in the project. > Seem pretty obvious, open an issue, people can subscribe or unsubscribe > on a per issue bassi and coordinate. I don't think that is reasonable. We have to start having people focusing and specializing and part of that is that more and more people will be involved in the project, but not coding. We will soon have people focused on design, fundraising, event planning, budget/admin, hiring, managing staff, industry relations, etc. We have a huge challenge in integrating these efforts into the code-centric core of the project, but we have to. > Even when you have your earplugs you also generate echo. > :-) Yes, that one was super weird - have never seen it before or since. Cheers, Brian > > -- > M > > -- > You received this message because you are subscribed to the Google Groups "Project Jupyter" group. > To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscribe at googlegroups.com. > To post to this group, send email to jupyter at googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE%40gmail.com. > For more options, visit https://groups.google.com/d/optout. -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From sylvain.corlay at gmail.com Sat Jun 27 23:46:40 2015 From: sylvain.corlay at gmail.com (Sylvain Corlay) Date: Sat, 27 Jun 2015 23:46:40 -0400 Subject: [IPython-dev] comm_info_[request/reply] : addition to the messaging protocol Message-ID: Hi All, I wanted to come back to the discussion on the request/reply mechanism for information on currently live comms that we proposed and was discussed at the last dev meeting. I would be happy to open an IPEP in the wiki on this subject. I am posting here because the mailing list is a less transient than gitter or the hangouts and it helps reaching the broader community that might not be closely following the GitHub repositories. Here is some context for those who are new to the subject: when a client is connected to an already-running kernel (for example in the case of a page reload in the notebook), it currently has no direct means of knowing what comms are open in the back-end. If we want it to be able to re-instantiate front-end counterpart to those comms, a way to retrieve this information must be implemented. In the case of the notebook, the front-end can try sending messages to some comm ids that the browser knows existed at some point because the model ids of the widget displayed in the notebook are saved in the local storage when the notebook is saved, but it clearly is insufficient and is essentially a stab in the dark. This information may be completely outdated. Hence a natural solution to that issue was to create a new shell message type, to request the currently open comms from the back-end. Because of the multi-repository nature of Jupyter, this resulted a the following PRs *jupyter-client:* https://github.com/jupyter/jupyter_client/pull/34 *ipykernel:* https://github.com/ipython/ipykernel/pull/25 (requires the previous PR) *notebook:* https://github.com/jupyter/notebook/pull/166 (requires the two previous PRs) *ipywidgets:* https://github.com/ipython/ipywidgets/pull/62 (requires the three previous PRs) My initial proposal was to return the list of valid ids because it is what is required for the use case of widgets. After the discussion in the dev meeting where Fernando was worried that the schema of the message might note have been the right one, I changed things a bit and called the messages *comm_info_[request/reply]*. The reply message a dictionary containing one key, called `comms`, which itself contains a dictionary of the form `*{comm_id : target_name}*` In the case of widgets, the target name is always ipython.widget but not necessarily in the general case. With this new schema, the whole information is available and other fields than `comms` could be added to the *comm_info_reply* message in the future. There are some workarounds for the lack of such a message in the protocol, one of which is implemented in the last commit in the PR for ipywidgets. However, I don't think that such contortions are the way to go. Feedback on the PRs is welcome. If other users of comms have ideas on this specific issue, please respond on the mailing list! Thanks, Sylvain -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sun Jun 28 00:17:14 2015 From: benjaminrk at gmail.com (MinRK) Date: Sun, 28 Jun 2015 05:17:14 +0100 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: I think user fridays, or something similar is a great idea. We tend to do our best work when we are *using* IPython most (I found a couple things to fix while teaching this week). I also think stabilizing and focusing on user features for a while is a good plan, and I think we can do that in many parts of IPython/Jupyter (remember, there isn't going to be a single IPython/Jupyter 5.0). Unfortunately, I think the client-side of the notebook is not a place where we can afford to do that right now. I think we need to almost start from scratch with that stuff notebook, in order to enable the things we want to do with it. Technically, we could do both: - create a 5.0 branch, where we start over with typescript/phosphor - continue to work on 4.1, 4.2, where we add these user-facing features with the code in the state that it is. This is actually one way that we can get more people working on the notebook - some people working on features in 4.1, 4.2, while the 5.x restructuring is going on. We don't need to follow the backports-only pattern for minor releases we have done for IPython recently, which we do in part due to our relative lack of developer time. In some ways, we are coming into the opposite problem - too many people, not enough available parallel work. I'm not saying it's necessarily what we should do, but it is an option available. -MinRK On Sat, Jun 27, 2015 at 10:55 AM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > Hi All, > > Thanks for doing this on the public Mailing list, it seem we are improving > our communication skills, > be careful though, the response start to have more and more implicit > knowledge. > > I do hear what Thomas is saying too, but I have seen to numerous time > people reimplementing > in javascript pieces of our codes because they are un-reusable (hydrogen, > notable mind, the > things that render notebook purely in js ... ) so it seem that **some** > refactoring is needed. > > I?m not really happy to have done the conversion to typescript without API > changes, > and not got my PRs merged, and it seemed like these last month any changes > to > the notebook has had a strong pushback. And without theses changes we > basically > won?t have any new features. That seem (to me) like a pretty stable > javascript Api > for something we said can change at any time as we **want** to refactor. > > Writing some of our SciPy tutorial also made me realize how some low level > feature > of the notebook API (I know, not stable) are broken in design, and > hopefully not used to much. > > So I really hope we clean things. > > > That being said, I would also really like to spend more time **using** the > notebook, > to do "user Friday? and show really cool things that can be done. Though > I think it is something > which is hard to do with the current technical dept. > > -- > M > > > > On Jun 27, 2015, at 08:30, Brian Granger wrote: > > > >> What's going to be very challenging in this effort is the *coordination* > >> part. We need to make sure that we have both a chance for proper > discussion > >> of the key ideas, and for open coordination of the effort while it's > >> happening, so that: > > > > Yep, I definitely agree that the coordination, organizational and > > communication aspects of the project will have to evolve in parallel > > to the code base. > > > >> > >> a) whoever ends up working on the critical path can focus on that and > just > >> get the nasty business done > >> > >> b) the rest of the team can feel comfortable that there's still other > areas > >> where work can proceed cleanly and without interference or blockage. > > > > My hope is that with the right technical choices and coordination, > > that we can remove the critical paths altogether. I have ideas on how > > to pull this off, and will post separately to the list about that. > > > >> It would be great if during scipy, you folks try to draft an outline of > what > >> the key pieces of that refactoring would entail, identifying what areas > of > >> the project would effectively hold locks. That would let us more easily > >> define what is NOT locked, so we can then partition things for everyone > else > >> to work on. > > > > Yes, I think that should be one of the main goals. But I want the > > discussion to be focused on *how* we do this, not *if*. > > > >> > >> Let's have that publicly documented (probably as an IPEP) so that we > don't > >> break the team in the process... > > > > Yep! > > > > Cheers, > > > > Brian > > > >> > >> Cheers > >> > >> f > >> > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > > > > > > > -- > > Brian E. Granger > > Cal Poly State University, San Luis Obispo > > @ellisonbg on Twitter and GitHub > > bgranger at calpoly.edu and ellisonbg at gmail.com > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Sun Jun 28 04:59:47 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Sun, 28 Jun 2015 10:59:47 +0200 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: Message-ID: <4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE@gmail.com> > On Jun 28, 2015, at 07:28, Brian Granger wrote: > > I like this idea. It would be great to encourage everyone involved in > the project to participate in this on a regular basis - even those who > are not "core developers." For example, having my students participate > would help integrate them into the larger project. > > I think we should limit the person time though - maybe 7 minutes? I proposed 10 min 3 weeks ago. I would also really appreciate to not have 3 words like: ?Tag?. An entry on the hackpad should be self explanatory (unless link to an issues which is self explanatory). Like if the person who wrote that is not here we can decide without them. Have 1 person responsible for taking notes, and turn the hangout into a written small report, that we (eventually) post on the blog to keep reader in the loop. That?s typically the kind of 1 way communication (this is what happened) that do not require general feedback from ML, except on relevant issues. >> - From this, we identify which sub-groups may later on break into other >> channels (our regular mailing list, issues on github, quick discussions on >> gitter, etc), to actually make decisions. > > We will probably have to think and talk further about the best way of > gathering the right sets of people at the right times and propagating > the information out, but this is a good next step. You mean we still don?t know how to open an issue on GitHub ? Seem pretty obvious, open an issue, people can subscribe or unsubscribe on a per issue bassi and coordinate. > I think it will be great to give this a shot. One of the ongoing > issues we have is with multiple people using the same mic. Even at > BIDS where you have a fantastic mic - it often leads to awful echoes > and audio problems. Even when you have your earplugs you also generate echo. :-) -- M From fperez.net at gmail.com Sat Jun 27 19:33:35 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 27 Jun 2015 16:33:35 -0700 Subject: [IPython-dev] Communications, mailing lists, gitter, etc... Message-ID: Hi all, sorry for the cross-post, but this is precisely about project-wide communication. Probably best to keep the thread on the Jupyter list. As the project has grown, we've tried to adapt our communications channels both to serve the community and to manage our limited resources. Time for email is scarce (I know I'm not the only one perennially behind), and for certain problems real-time conversation is extremely valuable. So we've added gitter to our mix of tools, and in some cases our gitter rooms are very useful. But we're also finding a problem: it's proving to be very hard to follow the "big picture" of the project if you're not directly participating in those conversations. Even for *me*, since these days I unfortunately rarely have that kind of time, many parts of the project have become quite opaque, and reading gitter logs is not the same as reading an email archive. While it's perfectly viable to read a long and complex email thread to catch up on a discussion after the fact, doing so with a real-time chat log is, in practice, nearly impossible. So, we'd like to fine-tune our communication practices, so that we hit a better balance of having a "slow record" over asynchronous email, while allowing for the more rapid-fire discussion that real-time channels like gitter allow. The tools at our disposal are: - Our mailing lists (jupyter & IPython-dev) - Our gitter rooms: we don't want to kill them, we just need to adjust what we want to use them for, and probably reduce the number of rooms. As the number of repos we have grows, having dozens of rooms is probably not manageable. - Github: issues will continue being used by folks as a mechanism to effectively submit problem reports that, in practice, are often questions. - StackOverflow: not to be underestimated, it's archival, searchable, etc. I don't want to dictate policy here, so I'll propose a rough first draft, to allow for discussion, ideas and fine-tuning: - We try to do a better job of communicating important ideas, questions, discussions, etc, on the project lists. Let's encourage our users to do the same. We also post key announcements more regularly here. Everyone should really feel comfortable - We stop pointing people to gitter as the *first* help stop. We should point them to the mailing list (and? OR?) StackOverflow instead. Having a non-archival medium be the first point of help means a vast amount of duplicated and wasted time helping people. - The Gitter real-time help room and repo rooms can remain active but we try to maintain a discipline of using them only for when folks need the real-time collaboration. If key decisions are made here, let's try to have a discipline of periodically posting a quick summary, even if it's just a very short one, to the mailing list, pointing out what happened. In the long run, it will help us all better keep in touch with what the project is doing. What other ideas do people have, that can help us better communicate, both among project regulars and with the more occasional, broader community participants? Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Sun Jun 28 01:28:20 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Sat, 27 Jun 2015 22:28:20 -0700 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: Message-ID: > - We turn our Tuesday 10am PST meeting into instead a "short reports" > meeting, where each person provides a quick update of the topics/problems > they've been working on, and presen?ts any questions they may have, help > they may need from others on the team, etc. I like this idea. It would be great to encourage everyone involved in the project to participate in this on a regular basis - even those who are not "core developers." For example, having my students participate would help integrate them into the larger project. I think we should limit the person time though - maybe 7 minutes? > > - From this, we identify which sub-groups may later on break into other > channels (our regular mailing list, issues on github, quick discussions on > gitter, etc), to actually make decisions. We will probably have to think and talk further about the best way of gathering the right sets of people at the right times and propagating the information out, but this is a good next step. > Then, there's the *separate* question of the technology for the meeting. > Lately, using Google+ has been driving us crazy, creating crashes for many > of us, dropping connections, etc... We'd like to experiment with other > options and see what works. > > For now, we're going to try BlueJeans, it's something we have at Berkeley, > and which allows both web browser and plain old phone client access, it may > be a little better than G+. We'll see how it goes. I think it will be great to give this a shot. One of the ongoing issues we have is with multiple people using the same mic. Even at BIDS where you have a fantastic mic - it often leads to awful echoes and audio problems. Cheers, Brian > > Thoughts? > > Cheers > > f > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > -- > You received this message because you are subscribed to the Google Groups > "Project Jupyter" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jupyter+unsubscribe at googlegroups.com. > To post to this group, send email to jupyter at googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jupyter/CAHAreOoFNhXpv9o_JrLUpUGuum60JUSW1JH8pQ873M0QEW%2BbMA%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From wes.turner at gmail.com Mon Jun 29 03:00:14 2015 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 29 Jun 2015 02:00:14 -0500 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: <4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE@gmail.com> Message-ID: * https://wrdrd.com/docs/consulting/software-development#stand-up-meeting * https://wrdrd.com/docs/consulting/software-development#digital-stand-up-meeting * etherpad-lite (operational transformation) * https://wrdrd.com/docs/consulting/team-building#collaboration-engineering On Jun 29, 2015 12:21 AM, "Brian Granger" wrote: > >> I think we should limit the person time though - maybe 7 minutes? > > > > I proposed 10 min 3 weeks ago. > > Yes and it was a good idea! The only problem is that 10 minutes is > incompatible with a meeting whole goal is to make a lot of technical > decisions. We found that out pretty quickly. > > Upon thinking more, I am thinking that putting a 5 minute max person > might even be too high. > > > > > I would also really appreciate to not have 3 words like: ?Tag?. An entry > on > > the hackpad should be self explanatory (unless link to an issues which > is self explanatory). > > Like if the person who wrote that is not here we can decide without them. > > But I think the main content of Fernando's proposal is that we won't > ever make decisions in the dev meetings. > > > > > Have 1 person responsible for taking notes, and turn the hangout into a > written small report, > > that we (eventually) post on the blog to keep reader in the loop. > > I don't think the blog post is appropriate for most of this stuff - > and I don't think we have anyone that wants to take the time to turn > the summary in to a weekly blog post yet. > > > >> We will probably have to think and talk further about the best way of > >> gathering the right sets of people at the right times and propagating > >> the information out, but this is a good next step. > > > > You mean we still don?t know how to open an issue on GitHub ? > > Of course not. But as the project grows, people will be working in a > more focused and specialized manner, unaware of what is going on in > the rest of the project. Even today, with ~15-20 repos, not all of us > are subscribed to all of them. We need ways of propagating information > from all that development activity on all of those repos in a way that > project leadership is able to guide the whole project in a coherent > manner. Also, remember, we are hiring a number of non-coders who will > need to participate in the project. > > > Seem pretty obvious, open an issue, people can subscribe or unsubscribe > > on a per issue bassi and coordinate. > > I don't think that is reasonable. We have to start having people > focusing and specializing and part of that is that more and more > people will be involved in the project, but not coding. We will soon > have people focused on design, fundraising, event planning, > budget/admin, hiring, managing staff, industry relations, etc. We have > a huge challenge in integrating these efforts into the code-centric > core of the project, but we have to. > > > Even when you have your earplugs you also generate echo. > > :-) > > Yes, that one was super weird - have never seen it before or since. > > Cheers, > > Brian > > > > > -- > > M > > > > -- > > You received this message because you are subscribed to the Google > Groups "Project Jupyter" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to jupyter+unsubscribe at googlegroups.com. > > To post to this group, send email to jupyter at googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/jupyter/4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE%40gmail.com > . > > For more options, visit https://groups.google.com/d/optout. > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sat Jun 27 23:07:55 2015 From: benjaminrk at gmail.com (MinRK) Date: Sun, 28 Jun 2015 04:07:55 +0100 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: Message-ID: I think turning Tuesdays into check-ins/reports is a good idea. Having just one or two topics for actual in-depth meetings with smaller groups should definitely be more effective. -MinRK On Sun, Jun 28, 2015 at 12:08 AM, Fernando Perez wrote: > Hi all, > > from looking at how our most recent dev meetings have been going, it > appears that we're probably reaching a size of team where having meetings > that are so long, and where we try to have detailed, back-and-forth > discussions for making technical decisions, is just not working well > anymore... > > While the idea of a public "all hands" dev meeting is one we're very > happy to have, and want to continue (and other projects seem to have > followed in our step), it seemed to work better when the group needing to > really discuss things in detail was a fair bit smaller. > > In recent meetings, we're having folks tune out quite a bit, and we're > probably not making very effective use of the time of others... > > So here's a proposal for a modified format: > > - We turn our Tuesday 10am PST meeting into instead a "short reports" > meeting, where each person provides a quick update of the topics/problems > they've been working on, and presen?ts any questions they may have, help > they may need from others on the team, etc. > > - From this, we identify which sub-groups may later on break into other > channels (our regular mailing list, issues on github, quick discussions on > gitter, etc), to actually make decisions. > > Basically, once a meeting has more than ~4-6 people, it doesn't really > make sense to have it really be a detailed, long technical discussion, > since most people will likely tune out. So only those people really > involved should participate. > > This way, the regular 'all hands' meeting will be a more useful way for > everyone to remain informed of what we're all doing on the project, how > things are going, etc. > > > Then, there's the *separate* question of the technology for the meeting. > Lately, using Google+ has been driving us crazy, creating crashes for many > of us, dropping connections, etc... We'd like to experiment with other > options and see what works. > > For now, we're going to try BlueJeans, it's something we have at Berkeley, > and which allows both web browser and plain old phone client access, it may > be a little better than G+. We'll see how it goes. > > Thoughts? > > Cheers > > f > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > -- > You received this message because you are subscribed to the Google Groups > "Project Jupyter" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jupyter+unsubscribe at googlegroups.com. > To post to this group, send email to jupyter at googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jupyter/CAHAreOoFNhXpv9o_JrLUpUGuum60JUSW1JH8pQ873M0QEW%2BbMA%40mail.gmail.com > > . > For more options, visit https://groups.google.com/d/optout. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Jun 29 03:59:19 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Mon, 29 Jun 2015 09:59:19 +0200 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: <4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE@gmail.com> Message-ID: <6C66D32A-B1E1-40D5-9BF3-E8E56D563F35@gmail.com> Hi Wes, > On Jun 29, 2015, at 09:00, Wes Turner wrote: > > * https://wrdrd.com/docs/consulting/software-development#stand-up-meeting > * https://wrdrd.com/docs/consulting/software-development#digital-stand-up-meeting > * etherpad-lite (operational transformation) > > * https://wrdrd.com/docs/consulting/team-building#collaboration-engineering Thanks for the links, can you, when you post link make a small recap ? It would avoid for those of us reading this offline to at least know what this is about, and also what the pint you are trying to make, as often people can interprete articles differently. Keep in mind that if you take 2 minutes to do that, and that people reading that only win 10 sec, but there is 100 of them, that still make a global win on 16 minutes for the project. Cf this small post of Nick Coghlan[1] explaining that 222 years of collective human existence pass each seconds. Tip: try to put all your links after your signature, if the mail do not make sense, then it probably won?t have enough impact. Also URL often disapear, so it make reading this in 1 year difficult. Thanks a lot, -- M [1] http://www.curiousefficiency.org/posts/2014/09/seven-billion-seconds-per-second.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Jun 27 19:08:10 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 27 Jun 2015 16:08:10 -0700 Subject: [IPython-dev] Experimenting with a modified dev meeting format? Message-ID: Hi all, from looking at how our most recent dev meetings have been going, it appears that we're probably reaching a size of team where having meetings that are so long, and where we try to have detailed, back-and-forth discussions for making technical decisions, is just not working well anymore... While the idea of a public "all hands" dev meeting is one we're very happy to have, and want to continue (and other projects seem to have followed in our step), it seemed to work better when the group needing to really discuss things in detail was a fair bit smaller. In recent meetings, we're having folks tune out quite a bit, and we're probably not making very effective use of the time of others... So here's a proposal for a modified format: - We turn our Tuesday 10am PST meeting into instead a "short reports" meeting, where each person provides a quick update of the topics/problems they've been working on, and presen?ts any questions they may have, help they may need from others on the team, etc. - From this, we identify which sub-groups may later on break into other channels (our regular mailing list, issues on github, quick discussions on gitter, etc), to actually make decisions. Basically, once a meeting has more than ~4-6 people, it doesn't really make sense to have it really be a detailed, long technical discussion, since most people will likely tune out. So only those people really involved should participate. This way, the regular 'all hands' meeting will be a more useful way for everyone to remain informed of what we're all doing on the project, how things are going, etc. Then, there's the *separate* question of the technology for the meeting. Lately, using Google+ has been driving us crazy, creating crashes for many of us, dropping connections, etc... We'd like to experiment with other options and see what works. For now, we're going to try BlueJeans, it's something we have at Berkeley, and which allows both web browser and plain old phone client access, it may be a little better than G+. We'll see how it goes. Thoughts? Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Jun 29 04:11:50 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Mon, 29 Jun 2015 10:11:50 +0200 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: > On Jun 29, 2015, at 06:03, Brian Granger wrote: > > I view the blog as being reserved for highly curated and copy-edited > content. I agree, though do you want to improve our communication or not ? Do you think there is not enough content for one Post per week? > That content is very time consuming to write and I don't want > to discourage smaller one-way announcements by telling people that > they all need to be blog posts. The question is do you think spending 1h announcing things **correctly**, is not worth the (potentially) infinite questions we will get after ? > Also the blog serves a much difference > audience - users - than a development focused mailing list. Once again, this depends on the content of the dev meeting, and the things we want to put on the blog. - Does the fact that we are going to SciPy matters ? - Does the fact that we split the repo matters ? - If we have any fundings matters ? Theses are topics that will touch both user and dev, and that I see us putting on the blog. The IPython user-ml and dev have been merge **because** the distinction user/dev is blurry. > For > development purposes, I don't see a significant difference between > announcements and 2 ways communications. Here is another metric to use > in deciding what is blog post worthy - do we want to tweet about it. > Most things on the mailing list - even when 1 way - don't pass that > test. We are starting to write the notebook using phosphor, there is a hydrogen plugin for atom connecting to jupyter, we are adding things to message spec, tartlets splits into tartlets and widgets tartlets are things I want to tweet about. But what do actually subscriber on this ML thinks ? -- M From wes.turner at gmail.com Mon Jun 29 04:14:13 2015 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 29 Jun 2015 03:14:13 -0500 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: <6C66D32A-B1E1-40D5-9BF3-E8E56D563F35@gmail.com> References: <4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE@gmail.com> <6C66D32A-B1E1-40D5-9BF3-E8E56D563F35@gmail.com> Message-ID: On Jun 29, 2015 3:01 AM, "Matthias Bussonnier" wrote: > > Hi Wes, > > >> On Jun 29, 2015, at 09:00, Wes Turner wrote: >> >> * https://wrdrd.com/docs/consulting/software-development#stand-up-meeting >> * https://wrdrd.com/docs/consulting/software-development#digital-stand-up-meeting >> >> * etherpad-lite (operational transformation) >> >> * https://wrdrd.com/docs/consulting/team-building#collaboration-engineering > > > Thanks for the links, can you, when you post link make a small recap ? Like minutes that persist from a digital stand up meeting? I could cc snippets of a source RST block? > It would avoid for those of us reading this offline to at least know what this > is about, and also what the pint you are trying to make, as often people can > interprete articles differently. Really, I noticed the discussion regarding three questions and stand-up meetings, and had just updated that section of the linked knowledge base of concept URIs (as well as https://wrdrd.com/docs/consulting/education-technology#jupyter-and-learning #jupyter-and-reproducibility) If I might interject, etherpad-lite is really the simplest possible way to write markdown for e.g. a github wiki as a team. > > Keep in mind that if you take 2 minutes to do that, and that people reading that > only win 10 sec, but there is 100 of them, that still make a global win on 16 minutes > for the project. > > Cf this small post of Nick Coghlan[1] explaining that 222 years of > collective human existence pass each seconds. > > Tip: try to put all your links after your signature, if the mail do not make sense, > then it probably won?t have enough impact. Also URL often disapear, so it make > reading this in 1 year difficult. > > Thanks a lot, > -- > M > > [1] http://www.curiousefficiency.org/posts/2014/09/seven-billion-seconds-per-second.html > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Jun 27 17:53:32 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 27 Jun 2015 14:53:32 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: On Sat, Jun 27, 2015 at 2:55 AM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > I do hear what Thomas is saying too, but I have seen to numerous time > people reimplementing > in javascript pieces of our codes because they are un-reusable (hydrogen, > notable mind, the > things that render notebook purely in js ... ) so it seem that **some** > refactoring is needed. > Yes, I really think we have no option but work on this... There's one word that keeps me up at night right now when I think of this: MySpace... Basically, we've established a lot of early momentum in the space of the notebook, our interactive kernel architecture, etc. But in doing so, we've actually shown the value of the problem space to be so important, that lots and lots of other players are now moving in, and many of them need to build new ideas rapidly. Fortunately for us, some of them are actually willing to work with us to refactor our machinery to enable a far better set of tools to be built in the near future atop of our same overall design. But unless we actually do that, most folks would eventually just decide to simply do something else and build elsewhere... So, let's try to not end up like MySpace. The refactoring work is really critical, and it's work we *can* do in a relatively limited timeframe. It won't be trivial, but it won't take a year either. With some careful planning and dedicated effort, we can pull through in a few months and have a cleaner foundation to build upon. Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Sun Jun 28 04:53:18 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Sun, 28 Jun 2015 10:53:18 +0200 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: > On Jun 28, 2015, at 01:33, Fernando Perez wrote: > > The tools at our disposal are: > > - Our mailing lists (jupyter & IPython-dev) > ... According to our website we also push people toward: - reddit/r/ipython - the wiki. > I don't want to dictate policy here, so I'll propose a rough first draft, to allow for discussion, ideas and fine-tuning: > > > - We try to do a better job of communicating important ideas, questions, discussions, etc, on the project lists. Let's encourage our users to do the same. We also post key announcements more regularly here. Everyone should really feel comfortable **please** please, try to distinguish one way communication from discussion on the ML. Announcement are hard to refer to to and google and the ML. Announcement are made for the blog. If you don?t expect that response are key to communication, use the blog, you can start a thread **About** a blog post, but [Ann] on ML hurt communication. > > - We stop pointing people to gitter as the *first* help stop. We should point them to the mailing list (and? OR?) StackOverflow instead. Having a non-archival medium be the first point of help means a vast amount of duplicated and wasted time helping people. Yes !!!! I would also appreciate decreasing the number of SO tags. > > - The Gitter real-time help room and repo rooms can remain active but we try to maintain a discipline of using them only for when folks need the real-time collaboration. If key decisions are made here, let's try to have a discipline of periodically posting a quick summary, even if it's just a very short one, to the mailing list, pointing out what happened. In the long run, it will help us all better keep in touch with what the project is doing. Also Yes ! > On Jun 28, 2015, at 08:04, Brian Granger wrote: > > I would add one point: > > When private conversations happen the participants post a short > summary to the mailing lists or github. I think that remains one of > the biggest communication challenges we have. Being geographically > isolated from UC Berkeley, I know this first hand from both sides. > There are times that the entire UC Berkeley team assumes Jon and I > know about something when we don't. And I am sure there are times that > Jon and I assume the Berkeley folks know things that we do. I also > have many phone conversations that I don't summarize well to the rest > of the team. In summary, I am as guilty as anyone on our team of not > summarizing private conversations to the rest of the team and > community and I want to work really hard to change this. Yes, same with our dev meetings. Wee should take more notes. > I would like to clarify which gitter channels we will use. Our current > set still feels a bit scattered. I would prefer to have one main > channel for each org and then only additional rooms for specific > projects that are very high traffic. We already said in hangout : ipython/ipython ipython/ipython/help Per project if a project deemed it necessary. (like hub as lots of traffic) + local privates ones, like the one where we ping each other for lunch, but that?s not relevant is it ? -- M From ellisonbg at gmail.com Sun Jun 28 02:04:01 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Sat, 27 Jun 2015 23:04:01 -0700 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: > The tools at our disposal are: > > - Our mailing lists (jupyter & IPython-dev) > > - Our gitter rooms: we don't want to kill them, we just need to adjust what > we want to use them for, and probably reduce the number of rooms. As the > number of repos we have grows, having dozens of rooms is probably not > manageable. > > - Github: issues will continue being used by folks as a mechanism to > effectively submit problem reports that, in practice, are often questions. > > - StackOverflow: not to be underestimated, it's archival, searchable, etc. One more tool we do have - live conversations (phone calls, skype, G+, etc). While these tools are not public, I know we will continue to use them - our acknowledging this will help us to integrate these into the public channels better. > - We try to do a better job of communicating important ideas, questions, > discussions, etc, on the project lists. Let's encourage our users to do the > same. We also post key announcements more regularly here. Everyone should > really feel comfortable I would add one point: When private conversations happen the participants post a short summary to the mailing lists or github. I think that remains one of the biggest communication challenges we have. Being geographically isolated from UC Berkeley, I know this first hand from both sides. There are times that the entire UC Berkeley team assumes Jon and I know about something when we don't. And I am sure there are times that Jon and I assume the Berkeley folks know things that we do. I also have many phone conversations that I don't summarize well to the rest of the team. In summary, I am as guilty as anyone on our team of not summarizing private conversations to the rest of the team and community and I want to work really hard to change this. > > - We stop pointing people to gitter as the *first* help stop. We should > point them to the mailing list (and? OR?) StackOverflow instead. Having a > non-archival medium be the first point of help means a vast amount of > duplicated and wasted time helping people. > > - The Gitter real-time help room and repo rooms can remain active but we try > to maintain a discipline of using them only for when folks need the > real-time collaboration. If key decisions are made here, let's try to have > a discipline of periodically posting a quick summary, even if it's just a > very short one, to the mailing list, pointing out what happened. In the > long run, it will help us all better keep in touch with what the project is > doing. > > > What other ideas do people have, that can help us better communicate, both > among project regulars and with the more occasional, broader community > participants? I would like to clarify which gitter channels we will use. Our current set still feels a bit scattered. I would prefer to have one main channel for each org and then only additional rooms for specific projects that are very high traffic. Would does everyone think? Cheers, Brian > > Cheers > > f > > > > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > -- > You received this message because you are subscribed to the Google Groups > "Project Jupyter" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jupyter+unsubscribe at googlegroups.com. > To post to this group, send email to jupyter at googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jupyter/CAHAreOpGXgRSj1vr8YzV9pdmpXCrZdE-tL%2BeVAV-Kr%2BVe7pkng%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From fperez.net at gmail.com Sat Jun 27 18:53:00 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 27 Jun 2015 15:53:00 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: Here's an email that Andrew sent me privately, but with his authorization, I'm replying to it on the public list as I think his concerns have a lot of merit and I'm sure others would be interested in the topic as well... ? On Fri, Jun 26, 2015 at 9:56 PM, Andrew Gibiansky < andrew.gibiansky at gmail.com> wrote: > Fernando, > > However, I would like to urge you and the IPython team to consider > focusing a little bit of effort towards something that has gone > unmentioned: *documentation*. > > The IPython documentation has suffered from a few main problems: > > - It is often out of date. The messaging spec that was accessible, as > of a few weeks ago, was out of date and simply wrong about some things in > IPython 3.1. > - Significant portions are undocumented, or documented only through > their original IPEPs. For example, the backbone widgets! I am currently > working with a student for GSOC to implement widget support for IHaskell, > and he regularly has to resort to spelunking through the IPython source to > implement the same functionality, rather than looking at a spec. > - Finally, and *most* importantly, the frontend is completely > undocumented. As far as I can tell, the suggested way towards developing > anything for the frontend is to grep through the source code, for example > for which events are available to be bound to. The result of this is that > developing frontend custom.js or extensions is nigh impossible; even simple > ones become a gargantuan task requiring you to delve deep into IPython JS > source code, and more extensive extensions can only be done by core IPython > contributors (or people with similarly deep knowledge). As someone who is > very familiar with IPython (having written IHaskell and a few frontend > extensions and customizations), it is very difficult even for me to get > anything done on the frontend, much less someone with less experience. I > believe this *significantly* hinders development of new extensions, > widgets, and frontends. > > Seeing Thomas' suggestion, I think it would be *really* great if the team > could tackle the occasionally poor documentation right now. (I say > 'occasionally' because the documentation is still much much better than > many other projects!) From an external standpoint, it seems like the amount > of technical debt in IPython is growing somewhat unmanageable to deal with > for an outsider, in part because of the lack of documentation. Even if now > is not the right time for a focus on documentation, it would be wonderful > if this were brought up and if perhaps it were scoped out as part of the > schedule for the upcoming months. > > Again, please take this in the best way possible ? this is not meant as > criticism of the way IPython is being developed but rather as an expression > of what would be important to me, a very frequent user of IPython and > developer of a fairly popular backend. With all of this in mind, I want to > say that I'm incredibly impressed with the quality and pace of development, > and it's exciting to see plans for 4.0 and 5.0 being made! > These concerns are absolutely spot-on, and we're painfully aware of the impact that the rather sad state of our docs has on users and developers of the system alike. Furthermore, we know that it creates a significant barrier to entry to new contributions, and to growing a healthier community of participation around the project. We have, sadly, dug a pretty deep hole here, and it will take some real effort to climb back out of it, so this isn't going to happen overnight. We've been, over the last few months, working very hard on trying to secure some resources to ensure that the core team can remain employed, which is an even more critical concern for some of us, as I hope you'll understand :) We hope to have some good news to report on that front before too long, and once we can get some proper resources in place, we actually will put documentation efforts as a first-class priority. Even before that, we have a few small steps in the right direction: - Min went this spring to the WriteTheDocs conference, to start meeting technical writers and get a bit more acquainted with that community, - Brian has already hired some new summer students who are starting to improve the layout of the websites. That's not the same as writing a bunch of new docs, but having better websites will help. The PRs are here: https://github.com/jupyter/design/issues/15, https://github.com/jupyter/design/issues/16, help always welcome. - We've actually already started work on restructuring all the repos for better docs, and I know that Thomas, Brian, Jess, Jon, Matthias and others (sorry folks for whoever I'm not crediting directly) have put work into it. At least now we have an overall skeleton so that the navigation of the docs for the overall project is more comprehensible: http://jupyter.readthedocs.org/en/latest/ and we're also working on tooling to smooth the integration of notebooks into the sphinx workflow. These efforts are all partial, and cooking at slow simmer because people are more or less at max capacity trying to get 4.0 out the door and with the Scipy conference breathing down our necks. I would say though that, once 4.0 is out the door, with this new skeleton in place, contributing docs will be a place where the community can *definitely* help the project. I know until now it was hard to even help with documentation, because the layout of our docs was so messy and monolithic. We're trying to precisely help with making it easier to *contribute*, please let us know if this new structure looks like a step in the right direction or not... But we are not, not at all, ignoring your concerns. Hopefully in a few days/weeks we'll have a bit more we can share on that front that will help too... Cheers, f -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick.bollweg at gmail.com Mon Jun 29 08:44:08 2015 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Mon, 29 Jun 2015 12:44:08 +0000 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: Wow, go away for a weekend and all hell breaks loose! Really good stuff here. +1 to docs living in notebooks, and using as their primary presentation, the notebook ui. When authoring a notebook, what if a user didn't have to leave the ui to go search five websites to find out a) what a particular class does and b) what other classes do that? For why, see the Mathematica documentation, which is really a better yardstick than other open source projects. We already maintain and ship to every user a sophisticated document management platform, the benefits of which are mostly destroyed when serialized to static html. Mix in indexed search, and wiki-style "what links here", and we've got something really powerful. If sphinx plays a role in exposing source files as notebooks, so much the better, but Mathematica, for example, has the burden of documenting everything for users in prose/signatures. I'm sure their developers write beautiful source for each other, as the platform is now something like 30 years old.... Can't be much debt left, right? Bringing in another thread, typescript will give us the descriptive power to provide deep, linked data links to browser code, and improve browser-side documentation generally. Phosphor then also has the ui abstracting power to make an in browser documentation viewer unsurprising. Additionally, kernel authors, kernel-agnostic nbextenders and such would then have a compelling reason to author, ship and maintain notebooks... A hook here and there, and you are indexed along with everything else. Something like ipymd could close the vcs maintainability gap (once it has metadata), while nosebook (once it does coverage, and in addition to other tests) could close the loop, making the shipping of uncovered/undocumented code a much more intentional act. Excited for more discussion! On 06:55, Mon, Jun 29, 2015 Fernando Perez wrote: > Here's an email that Andrew sent me privately, but with his authorization, > I'm replying to it on the public list as I think his concerns have a lot of > merit and I'm sure others would be interested in the topic as well... > ? > > On Fri, Jun 26, 2015 at 9:56 PM, Andrew Gibiansky < > andrew.gibiansky at gmail.com> wrote: > >> Fernando, >> >> However, I would like to urge you and the IPython team to consider >> focusing a little bit of effort towards something that has gone >> unmentioned: *documentation*. >> >> The IPython documentation has suffered from a few main problems: >> >> - It is often out of date. The messaging spec that was accessible, as >> of a few weeks ago, was out of date and simply wrong about some things in >> IPython 3.1. >> - Significant portions are undocumented, or documented only through >> their original IPEPs. For example, the backbone widgets! I am currently >> working with a student for GSOC to implement widget support for IHaskell, >> and he regularly has to resort to spelunking through the IPython source to >> implement the same functionality, rather than looking at a spec. >> - Finally, and *most* importantly, the frontend is completely >> undocumented. As far as I can tell, the suggested way towards developing >> anything for the frontend is to grep through the source code, for example >> for which events are available to be bound to. The result of this is that >> developing frontend custom.js or extensions is nigh impossible; even simple >> ones become a gargantuan task requiring you to delve deep into IPython JS >> source code, and more extensive extensions can only be done by core IPython >> contributors (or people with similarly deep knowledge). As someone who is >> very familiar with IPython (having written IHaskell and a few frontend >> extensions and customizations), it is very difficult even for me to get >> anything done on the frontend, much less someone with less experience. I >> believe this *significantly* hinders development of new extensions, >> widgets, and frontends. >> >> Seeing Thomas' suggestion, I think it would be *really* great if the >> team could tackle the occasionally poor documentation right now. (I say >> 'occasionally' because the documentation is still much much better than >> many other projects!) From an external standpoint, it seems like the amount >> of technical debt in IPython is growing somewhat unmanageable to deal with >> for an outsider, in part because of the lack of documentation. Even if now >> is not the right time for a focus on documentation, it would be wonderful >> if this were brought up and if perhaps it were scoped out as part of the >> schedule for the upcoming months. >> >> Again, please take this in the best way possible ? this is not meant as >> criticism of the way IPython is being developed but rather as an expression >> of what would be important to me, a very frequent user of IPython and >> developer of a fairly popular backend. With all of this in mind, I want to >> say that I'm incredibly impressed with the quality and pace of development, >> and it's exciting to see plans for 4.0 and 5.0 being made! >> > > These concerns are absolutely spot-on, and we're painfully aware of the > impact that the rather sad state of our docs has on users and developers of > the system alike. Furthermore, we know that it creates a significant > barrier to entry to new contributions, and to growing a healthier community > of participation around the project. > > We have, sadly, dug a pretty deep hole here, and it will take some real > effort to climb back out of it, so this isn't going to happen overnight. > > We've been, over the last few months, working very hard on trying to > secure some resources to ensure that the core team can remain employed, > which is an even more critical concern for some of us, as I hope you'll > understand :) > > We hope to have some good news to report on that front before too long, > and once we can get some proper resources in place, we actually will put > documentation efforts as a first-class priority. Even before that, we have > a few small steps in the right direction: > > - Min went this spring to the WriteTheDocs conference, to start meeting > technical writers and get a bit more acquainted with that community, > > - Brian has already hired some new summer students who are starting to > improve the layout of the websites. That's not the same as writing a bunch > of new docs, but having better websites will help. The PRs are here: > https://github.com/jupyter/design/issues/15, > https://github.com/jupyter/design/issues/16, help always welcome. > > - We've actually already started work on restructuring all the repos for > better docs, and I know that Thomas, Brian, Jess, Jon, Matthias and others > (sorry folks for whoever I'm not crediting directly) have put work into it. > At least now we have an overall skeleton so that the navigation of the > docs for the overall project is more comprehensible: > > http://jupyter.readthedocs.org/en/latest/ > > and we're also working on tooling to smooth the integration of notebooks > into the sphinx workflow. > > > These efforts are all partial, and cooking at slow simmer because people > are more or less at max capacity trying to get 4.0 out the door and with > the Scipy conference breathing down our necks. > > I would say though that, once 4.0 is out the door, with this new skeleton > in place, contributing docs will be a place where the community can > *definitely* help the project. I know until now it was hard to even help > with documentation, because the layout of our docs was so messy and > monolithic. We're trying to precisely help with making it easier to > *contribute*, please let us know if this new structure looks like a step in > the right direction or not... > > > But we are not, not at all, ignoring your concerns. Hopefully in a few > days/weeks we'll have a bit more we can share on that front that will help > too... > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick.bollweg at gmail.com Mon Jun 29 08:58:42 2015 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Mon, 29 Jun 2015 12:58:42 +0000 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: <4D2EC1FC-C333-4DCF-B8DE-9D005FE934FE@gmail.com> <6C66D32A-B1E1-40D5-9BF3-E8E56D563F35@gmail.com> Message-ID: Status reports in lighting talk format as slide notebooks! Mmm, dog food. On 04:17, Mon, Jun 29, 2015 Wes Turner wrote: > > On Jun 29, 2015 3:01 AM, "Matthias Bussonnier" < > bussonniermatthias at gmail.com> wrote: > > > > Hi Wes, > > > > > >> On Jun 29, 2015, at 09:00, Wes Turner wrote: > >> > >> * > https://wrdrd.com/docs/consulting/software-development#stand-up-meeting > >> * > https://wrdrd.com/docs/consulting/software-development#digital-stand-up-meeting > >> > >> * etherpad-lite (operational transformation) > >> > >> * > https://wrdrd.com/docs/consulting/team-building#collaboration-engineering > > > > > > Thanks for the links, can you, when you post link make a small recap ? > > Like minutes that persist from a digital stand up meeting? > > I could cc snippets of a source RST block? > > > It would avoid for those of us reading this offline to at least know > what this > > is about, and also what the pint you are trying to make, as often people > can > > interprete articles differently. > > Really, I noticed the discussion regarding three questions and stand-up > meetings, and had just updated that section of the linked knowledge base of > concept URIs (as well as > https://wrdrd.com/docs/consulting/education-technology#jupyter-and-learning > #jupyter-and-reproducibility) > > If I might interject, etherpad-lite is really the simplest possible way to > write markdown for e.g. a github wiki as a team. > > > > > Keep in mind that if you take 2 minutes to do that, and that people > reading that > > only win 10 sec, but there is 100 of them, that still make a global win > on 16 minutes > > for the project. > > > > Cf this small post of Nick Coghlan[1] explaining that 222 years of > > collective human existence pass each seconds. > > > > Tip: try to put all your links after your signature, if the mail do not > make sense, > > then it probably won?t have enough impact. Also URL often disapear, so > it make > > reading this in 1 year difficult. > > > > Thanks a lot, > > -- > > M > > > > [1] > http://www.curiousefficiency.org/posts/2014/09/seven-billion-seconds-per-second.html > > > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason at jasongrout.org Mon Jun 29 09:25:31 2015 From: jason at jasongrout.org (Jason Grout) Date: Mon, 29 Jun 2015 09:25:31 -0400 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: <5591474B.3060506@jasongrout.org> On 6/26/15 19:45, Fernando Perez wrote: > While I hear very much the spirit of what you are saying, and I > certainly think that we can't lose sight that the *only* thing that > ultimately matters is whether we serve our users well or not, there's a > big piece that is already burning under us that probably can't wait. In > fact, at the last dev meeting, Jason already posted his new draft code > in this direction: > > https://github.com/jasongrout/phosphor-notebook I just wanted to mention that I support what Fernando, Brian, and Chris have said about moving forward with refactoring the notebook. We're making good progress, even while still ramping up. For example, Steven Silvester has put a lot of work recently in porting over the kernel javascript to Typescript and phosphor (along with dependencies): https://github.com/jasongrout/phosphor-notebook/pull/2 I just put in an in-progress pull request for documenting the API for kernels, kernelspecs, and sessions (which I realized when looking at the kernel javascript file was woefully undocumented/incorrectly documented): https://github.com/jupyter/notebook/pull/173. This shows our refactoring work is also having an immediate direct impact on the current notebook as well. In another message on this thread, Min suggested having a 5.x branch for further development, like the phosphor notebook. For now, I think the phosphor notebook can proceed as a separate project - it's totally a front-end thing at this point, and we're doing enough code clean-up and rewriting from js to typescript that I think it's all right to start in a fresh repo. Which brings up another point: can we make an official Jupyter repo for the phosphor notebook work, rather than using my personal repo? I'm happy to continue hosting https://github.com/jasongrout/phosphor-notebook/ in my personal github account for the time being, or set up a temporary organization so we can collaborate more effectively, but I think it would make more sense to bump it up to an experimental repo in the jupyter github organization, developed in parallel with the current notebook. Thomas, one thing to consider is that us working on a phosphor notebook doesn't preclude interested people from enhancing the existing notebook in the short term. We'd like the phosphor notebook to get to a comparable state with the current notebook as quickly as possible, but it will still take some time. Also, I totally agree with Thomas that dogfooding the notebook (and watching/helping others actually use it to get work done) is *extremely* important to understanding what we want here. And I also agree with others on this thread that documentation is sorely lacking. We'll be working on that in the phosphor notebook as we go along too. Thanks, Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Jun 29 09:43:54 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Mon, 29 Jun 2015 15:43:54 +0200 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: <5591474B.3060506@jasongrout.org> References: <5591474B.3060506@jasongrout.org> Message-ID: <4EDF5922-A4A8-49DB-A1AB-E9C3951EE83B@gmail.com> > > In another message on this thread, Min suggested having a 5.x branch for further development, like the phosphor notebook. For now, I think the phosphor notebook can proceed as a separate project - it's totally a front-end thing at this point, and we're doing enough code clean-up and rewriting from js to typescript that I think it's all right to start in a fresh repo. Which brings up another point: can we make an official Jupyter repo for the phosphor notebook work, rather than using my personal repo? I'm happy to continue hosting https://github.com/jasongrout/phosphor-notebook/ in my personal github account for the time being, or set up a temporary organization so we can collaborate more effectively, but I think it would make more sense to bump it up to an experimental repo in the jupyter github organization, developed in parallel with the current notebook. Done, Jason you are admin, I **think** you can all people to the team. I?m hopping on my plane, like in 5 min. Feeling like Linus writing a rc release not from airports as often. (I haven?t forked your to make sure it appears as a root repo) Quick +1 on typescript the type aliases allow you to so define Path extends String {} Which auto produce nice docs with meaning instead of just foo(a:string, a:string) you get foo(a:Path, b:NotebookName) which is really good for docs. See you Tuesday morning on dev meeting, which for once should **not** be public, as we are trying BlueJeans. -- M > > Thomas, one thing to consider is that us working on a phosphor notebook doesn't preclude interested people from enhancing the existing notebook in the short term. We'd like the phosphor notebook to get to a comparable state with the current notebook as quickly as possible, but it will still take some time. > > Also, I totally agree with Thomas that dogfooding the notebook (and watching/helping others actually use it to get work done) is *extremely* important to understanding what we want here. And I also agree with others on this thread that documentation is sorely lacking. We'll be working on that in the phosphor notebook as we go along too. > > Thanks, > > Jason > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgbkrk at gmail.com Mon Jun 29 09:59:47 2015 From: rgbkrk at gmail.com (Kyle Kelley) Date: Mon, 29 Jun 2015 08:59:47 -0500 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: <4EDF5922-A4A8-49DB-A1AB-E9C3951EE83B@gmail.com> References: <5591474B.3060506@jasongrout.org> <4EDF5922-A4A8-49DB-A1AB-E9C3951EE83B@gmail.com> Message-ID: Gosh, I'd rather read foo(path:string, notebookName:string) any day, instead of having to *also* reference aliases infer based on the name itself. On Mon, Jun 29, 2015 at 8:43 AM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > > In another message on this thread, Min suggested having a 5.x branch for > further development, like the phosphor notebook. For now, I think the > phosphor notebook can proceed as a separate project - it's totally a > front-end thing at this point, and we're doing enough code clean-up and > rewriting from js to typescript that I think it's all right to start in a > fresh repo. Which brings up another point: can we make an official > Jupyter repo for the phosphor notebook work, rather than using my personal > repo? I'm happy to continue hosting > https://github.com/jasongrout/phosphor-notebook/ in my personal github > account for the time being, or set up a temporary organization so we can > collaborate more effectively, but I think it would make more sense to bump > it up to an experimental repo in the jupyter github organization, developed > in parallel with the current notebook. > > > Done, Jason you are admin, I **think** you can all people to the team. > I?m hopping on my plane, like in 5 min. Feeling like Linus writing a rc > release not from airports as often. > (I haven?t forked your to make sure it appears as a root repo) > > > Quick +1 on typescript the type aliases allow you to so > > define Path extends String {} > > Which auto produce nice docs with meaning instead of just foo(a:string, > a:string) you get foo(a:Path, b:NotebookName) > which is really good for docs. > > > See you Tuesday morning on dev meeting, which for once should **not** be > public, as we are trying BlueJeans. > -- > M > > > Thomas, one thing to consider is that us working on a phosphor notebook > doesn't preclude interested people from enhancing the existing notebook in > the short term. We'd like the phosphor notebook to get to a comparable > state with the current notebook as quickly as possible, but it will still > take some time. > > Also, I totally agree with Thomas that dogfooding the notebook (and > watching/helping others actually use it to get work done) is **extremely** > important to understanding what we want here. And I also agree with others > on this thread that documentation is sorely lacking. We'll be working on > that in the phosphor notebook as we go along too. > > Thanks, > > Jason > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Kyle Kelley (@rgbkrk ; lambdaops.com, developer.rackspace.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylvain.corlay at gmail.com Mon Jun 29 10:04:45 2015 From: sylvain.corlay at gmail.com (Sylvain Corlay) Date: Mon, 29 Jun 2015 10:04:45 -0400 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: <5591474B.3060506@jasongrout.org> References: <5591474B.3060506@jasongrout.org> Message-ID: I wanted to +1 the proposal to start creating branches for new versions when a feature freeze occurs. Independently of the discussion on phosphor, I completely agree with Min on the diagnosis that there is not enough available parallel work. Regarding phosphor and the work on refactoring the front-end, thanks for creating the centralized phosphor notebook repository in the organization. I did some experiments lately with the widgets and did not know where this could fall, or how to share it without requiring it to install phosphor etc. Coordination is also important for new developments, even when they have not yet achieved the stability of the main components of the project. Best, Sylvain On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout wrote: > On 6/26/15 19:45, Fernando Perez wrote: > > While I hear very much the spirit of what you are saying, and I > certainly think that we can't lose sight that the **only** thing that > ultimately matters is whether we serve our users well or not, there's a > big piece that is already burning under us that probably can't wait. In > fact, at the last dev meeting, Jason already posted his new draft code > in this direction: > > https://github.com/jasongrout/phosphor-notebook > > > I just wanted to mention that I support what Fernando, Brian, and Chris > have said about moving forward with refactoring the notebook. We're making > good progress, even while still ramping up. For example, Steven Silvester > has put a lot of work recently in porting over the kernel javascript to > Typescript and phosphor (along with dependencies): > > https://github.com/jasongrout/phosphor-notebook/pull/2 > > I just put in an in-progress pull request for documenting the API for > kernels, kernelspecs, and sessions (which I realized when looking at the > kernel javascript file was woefully undocumented/incorrectly documented): > https://github.com/jupyter/notebook/pull/173. This shows our refactoring > work is also having an immediate direct impact on the current notebook as > well. > > In another message on this thread, Min suggested having a 5.x branch for > further development, like the phosphor notebook. For now, I think the > phosphor notebook can proceed as a separate project - it's totally a > front-end thing at this point, and we're doing enough code clean-up and > rewriting from js to typescript that I think it's all right to start in a > fresh repo. Which brings up another point: can we make an official > Jupyter repo for the phosphor notebook work, rather than using my personal > repo? I'm happy to continue hosting > https://github.com/jasongrout/phosphor-notebook/ in my personal github > account for the time being, or set up a temporary organization so we can > collaborate more effectively, but I think it would make more sense to bump > it up to an experimental repo in the jupyter github organization, developed > in parallel with the current notebook. > > Thomas, one thing to consider is that us working on a phosphor notebook > doesn't preclude interested people from enhancing the existing notebook in > the short term. We'd like the phosphor notebook to get to a comparable > state with the current notebook as quickly as possible, but it will still > take some time. > > Also, I totally agree with Thomas that dogfooding the notebook (and > watching/helping others actually use it to get work done) is **extremely** > important to understanding what we want here. And I also agree with others > on this thread that documentation is sorely lacking. We'll be working on > that in the phosphor notebook as we go along too. > > Thanks, > > Jason > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Mon Jun 29 12:04:42 2015 From: steve at holdenweb.com (Steve Holden) Date: Mon, 29 Jun 2015 17:04:42 +0100 Subject: [IPython-dev] Notebooks enforce specific kernel Message-ID: Since this may be a configuration issue I thought I'd ask on the list before filing an issue. I created a Python2 virtual environment and installed IPython[notebook] using pip, plus scipy and some other stuff. I then opened an existing notebook, only to find on executing it that it claims scipy isn't installed. I noticed that the notebook had automatically opened with a Python 3 kernel, which I found very odd. sys.prefix tells me that it's running Python 3 from a completely unconnected virtual environment. Now I did, once upon a time, attempt to install a Python2 kernel, but my configuration control isn't good enough to remind me what I did. I can't trace any reference to the "unconnected virtual environment" anywhere in my ~/.ipython directory. This raises two questions: 1: Can anyone tell me where I may have installed this rogue kernel; and 2: It seems notebooks contain a kernel specification of sorts. Does this mean I can't write a notebook that will run in both Python 2 and Python 3? regards Steve -- Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / @holdenweb -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Mon Jun 29 12:14:34 2015 From: steve at holdenweb.com (Steve Holden) Date: Mon, 29 Jun 2015 17:14:34 +0100 Subject: [IPython-dev] Notebooks enforce specific kernel In-Reply-To: References: Message-ID: Inevitably, almost as soon as I had sent the message I found and deleted kernel specs in /usr/local/share/jupyter, which seems to solve the problem (as well as explain the message about "attempting to work in a virtual environment, as one of the specs was indeed pointing to en executable in a virtualenv). regards Steve On Jun 29, 2015, at 5:04 PM, Steve Holden wrote: > Since this may be a configuration issue I thought I'd ask on the list before filing an issue. > > I created a Python2 virtual environment and installed IPython[notebook] using pip, plus scipy and some other stuff. I then opened an existing notebook, only to find on executing it that it claims scipy isn't installed. > > I noticed that the notebook had automatically opened with a Python 3 kernel, which I found very odd. sys.prefix tells me that it's running Python 3 from a completely unconnected virtual environment. > > Now I did, once upon a time, attempt to install a Python2 kernel, but my configuration control isn't good enough to remind me what I did. I can't trace any reference to the "unconnected virtual environment" anywhere in my ~/.ipython directory. This raises two questions: > > 1: Can anyone tell me where I may have installed this rogue kernel; and > > 2: It seems notebooks contain a kernel specification of sorts. Does this mean I can't write a notebook that will run in both Python 2 and Python 3? > > regards > Steve > -- > Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / @holdenweb > > > > > -- Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / @holdenweb -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Mon Jun 29 13:24:30 2015 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 29 Jun 2015 12:24:30 -0500 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: On Jun 29, 2015 5:39 AM, "Brian Granger" wrote: > > > The tools at our disposal are: > > > > - Our mailing lists (jupyter & IPython-dev) > > > > - Our gitter rooms: we don't want to kill them, we just need to adjust what > > we want to use them for, and probably reduce the number of rooms. As the > > number of repos we have grows, having dozens of rooms is probably not > > manageable. > > > > - Github: issues will continue being used by folks as a mechanism to > > effectively submit problem reports that, in practice, are often questions. > > > > - StackOverflow: not to be underestimated, it's archival, searchable, etc. > > One more tool we do have - live conversations (phone calls, skype, G+, > etc). While these tools are not public, I know we will continue to > use them - our acknowledging this will help us to integrate these into > the public channels better. > > > - We try to do a better job of communicating important ideas, questions, > > discussions, etc, on the project lists. Let's encourage our users to do the > > same. We also post key announcements more regularly here. Everyone should > > really feel comfortable > > I would add one point: > > When private conversations happen the participants post a short > summary to the mailing lists or github. I think that remains one of > the biggest communication challenges we have. Being geographically > isolated from UC Berkeley, I know this first hand from both sides. > There are times that the entire UC Berkeley team assumes Jon and I > know about something when we don't. And I am sure there are times that > Jon and I assume the Berkeley folks know things that we do. I also > have many phone conversations that I don't summarize well to the rest > of the team. In summary, I am as guilty as anyone on our team of not > summarizing private conversations to the rest of the team and > community and I want to work really hard to change this. > > > > > - We stop pointing people to gitter as the *first* help stop. We should > > point them to the mailing list (and? OR?) StackOverflow instead. Having a > > non-archival medium be the first point of help means a vast amount of > > duplicated and wasted time helping people. > > > > - The Gitter real-time help room and repo rooms can remain active but we try > > to maintain a discipline of using them only for when folks need the > > real-time collaboration. If key decisions are made here, let's try to have > > a discipline of periodically posting a quick summary, even if it's just a > > very short one, to the mailing list, pointing out what happened. In the > > long run, it will help us all better keep in touch with what the project is > > doing. > > > > > > What other ideas do people have, that can help us better communicate, both > > among project regulars and with the more occasional, broader community > > participants? > > I would like to clarify which gitter channels we will use. Our current > set still feels a bit scattered. I would prefer to have one main > channel for each org and then only additional rooms for specific > projects that are very high traffic. > > Would does everyone think? That makes sense. Links to docs, sources, issues? Someone could get all of these IPython/Jupyter link patterns together? https://westurner.org/wiki/ideas#open-source-mailing-list-extractor (and then make vague references to documentation and issues #3 with no links)? These are all valued channels with different searchability; and linking with URIs. > > Cheers, > > Brian > > > > > Cheers > > > > f > > > > > > > > > > -- > > Fernando Perez (@fperez_org; http://fperez.org) > > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > > fernando.perez-at-berkeley: contact me here for any direct mail > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Project Jupyter" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to jupyter+unsubscribe at googlegroups.com. > > To post to this group, send email to jupyter at googlegroups.com. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/jupyter/CAHAreOpGXgRSj1vr8YzV9pdmpXCrZdE-tL%2BeVAV-Kr%2BVe7pkng%40mail.gmail.com . > > For more options, visit https://groups.google.com/d/optout. > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Jun 29 13:41:15 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 29 Jun 2015 10:41:15 -0700 Subject: [IPython-dev] Notebooks enforce specific kernel In-Reply-To: References: Message-ID: There's two ways you can use kernels with virtualenvs: 1. Don't install kernelspecs, launch the notebook inside virtualenvs when you want them. 2. Create kernelspecs referring to each virtualenv so you can switch between them in the notebook interface. Unfortunately, the two don't mix very well, so as you found, once you've installed a kernelspec it may end up using that instead of the virtualenv where you started the notebook. > 2: It seems notebooks contain a kernel specification of sorts. Does this mean I can't write a notebook that will run in both Python 2 and Python 3? If the user only has one Python kernelspec installed, that will be automatically selected whether the notebook specifies Python 3 or 2. If they have both, it will prefer the one it was saved with. However, there isn't currently any way to declare in the metadata that a notebook works with both, or with only one. Thomas On 29 June 2015 at 09:14, Steve Holden wrote: > Inevitably, almost as soon as I had sent the message I found and deleted > kernel specs in /usr/local/share/jupyter, which seems to solve the problem > (as well as explain the message about "attempting to work in a virtual > environment, as one of the specs was indeed pointing to en executable in a > virtualenv). > > regards > Steve > > On Jun 29, 2015, at 5:04 PM, Steve Holden wrote: > > Since this may be a configuration issue I thought I'd ask on the list > before filing an issue. > > I created a Python2 virtual environment and installed IPython[notebook] > using pip, plus scipy and some other stuff. I then opened an existing > notebook, only to find on executing it that it claims scipy isn't installed. > > I noticed that the notebook had automatically opened with a Python 3 > kernel, which I found very odd. sys.prefix tells me that it's running > Python 3 from a completely unconnected virtual environment. > > Now I did, once upon a time, attempt to install a Python2 kernel, but my > configuration control isn't good enough to remind me what I did. I can't > trace any reference to the "unconnected virtual environment" anywhere in my > ~/.ipython directory. This raises two questions: > > 1: Can anyone tell me where I may have installed this rogue kernel; and > > 2: It seems notebooks contain a kernel specification of sorts. Does this > mean I can't write a notebook that will run in both Python 2 and Python 3? > > regards > Steve > -- > Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 > / @holdenweb > > > > > > > -- > Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 > / @holdenweb > > > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon.freder at gmail.com Mon Jun 29 13:54:54 2015 From: jon.freder at gmail.com (Jonathan Frederic) Date: Mon, 29 Jun 2015 10:54:54 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: > > I wanted to +1 the proposal to start creating branches for new versions > when a feature freeze occurs. +1 here too, I'm going to do that with ipywidgets for SciPy, create a 5.x branch. There's a lot of stuff I want to get a jump start on, and SciPy is a great time to do it. I don't want it to end up like numerous other experiments, which end up getting thrown out and redone completely just because of stagmentation and rebase difficulty. WRT to the documentation debt, I think it's important to remind everyone that that is intentional! I've looked at adding JS docs a couple times now, but when I brought it up we've decided as a group that it was lower priority because we did not want to commit JavaScript APIs that we know will change. I think when we figure out how the front-end packaging and component refactor will work, we definitely want to commit to **something**. On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay wrote: > I wanted to +1 the proposal to start creating branches for new versions > when a feature freeze occurs. Independently of the discussion on phosphor, > I completely agree with Min on the diagnosis that there is not enough > available parallel work. > > Regarding phosphor and the work on refactoring the front-end, thanks for > creating the centralized phosphor notebook repository in the organization. > I did some experiments lately with the widgets and did not know where this > could fall, or how to share it without requiring it to install phosphor > etc. Coordination is also important for new developments, even when they > have not yet achieved the stability of the main components of the project. > > Best, > > Sylvain > > > > On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout wrote: > >> On 6/26/15 19:45, Fernando Perez wrote: >> >> While I hear very much the spirit of what you are saying, and I >> certainly think that we can't lose sight that the **only** thing that >> ultimately matters is whether we serve our users well or not, there's a >> big piece that is already burning under us that probably can't wait. In >> fact, at the last dev meeting, Jason already posted his new draft code >> in this direction: >> >> https://github.com/jasongrout/phosphor-notebook >> >> >> I just wanted to mention that I support what Fernando, Brian, and Chris >> have said about moving forward with refactoring the notebook. We're making >> good progress, even while still ramping up. For example, Steven Silvester >> has put a lot of work recently in porting over the kernel javascript to >> Typescript and phosphor (along with dependencies): >> >> https://github.com/jasongrout/phosphor-notebook/pull/2 >> >> I just put in an in-progress pull request for documenting the API for >> kernels, kernelspecs, and sessions (which I realized when looking at the >> kernel javascript file was woefully undocumented/incorrectly documented): >> https://github.com/jupyter/notebook/pull/173. This shows our >> refactoring work is also having an immediate direct impact on the current >> notebook as well. >> >> In another message on this thread, Min suggested having a 5.x branch for >> further development, like the phosphor notebook. For now, I think the >> phosphor notebook can proceed as a separate project - it's totally a >> front-end thing at this point, and we're doing enough code clean-up and >> rewriting from js to typescript that I think it's all right to start in a >> fresh repo. Which brings up another point: can we make an official >> Jupyter repo for the phosphor notebook work, rather than using my personal >> repo? I'm happy to continue hosting >> https://github.com/jasongrout/phosphor-notebook/ in my personal github >> account for the time being, or set up a temporary organization so we can >> collaborate more effectively, but I think it would make more sense to bump >> it up to an experimental repo in the jupyter github organization, developed >> in parallel with the current notebook. >> >> Thomas, one thing to consider is that us working on a phosphor notebook >> doesn't preclude interested people from enhancing the existing notebook in >> the short term. We'd like the phosphor notebook to get to a comparable >> state with the current notebook as quickly as possible, but it will still >> take some time. >> >> Also, I totally agree with Thomas that dogfooding the notebook (and >> watching/helping others actually use it to get work done) is >> **extremely** important to understanding what we want here. And I also >> agree with others on this thread that documentation is sorely lacking. >> We'll be working on that in the phosphor notebook as we go along too. >> >> Thanks, >> >> Jason >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Jun 29 13:54:52 2015 From: benjaminrk at gmail.com (MinRK) Date: Mon, 29 Jun 2015 10:54:52 -0700 Subject: [IPython-dev] Notebooks enforce specific kernel In-Reply-To: References: Message-ID: I realize there's an issue with IPython's attempts to pseudo-activate a virtualenv if one is found but IPython isn't using it. If the notebook server is in an env and the kernel isn't, the kernel will try to activate the server's env (right?). Should we: 1. remove the attempts to activate envs 2. pop VIRTUAL_ENV from the env passed down to kernels (either or both)? -MinRK On Mon, Jun 29, 2015 at 10:41 AM, Thomas Kluyver wrote: > There's two ways you can use kernels with virtualenvs: > > 1. Don't install kernelspecs, launch the notebook inside virtualenvs when > you want them. > 2. Create kernelspecs referring to each virtualenv so you can switch > between them in the notebook interface. > > Unfortunately, the two don't mix very well, so as you found, once you've > installed a kernelspec it may end up using that instead of the virtualenv > where you started the notebook. > > > 2: It seems notebooks contain a kernel specification of sorts. Does this > mean I can't write a notebook that will run in both Python 2 and Python 3? > > If the user only has one Python kernelspec installed, that will be > automatically selected whether the notebook specifies Python 3 or 2. If > they have both, it will prefer the one it was saved with. However, there > isn't currently any way to declare in the metadata that a notebook works > with both, or with only one. > > Thomas > > On 29 June 2015 at 09:14, Steve Holden wrote: > >> Inevitably, almost as soon as I had sent the message I found and deleted >> kernel specs in /usr/local/share/jupyter, which seems to solve the problem >> (as well as explain the message about "attempting to work in a virtual >> environment, as one of the specs was indeed pointing to en executable in a >> virtualenv). >> >> regards >> Steve >> >> On Jun 29, 2015, at 5:04 PM, Steve Holden wrote: >> >> Since this may be a configuration issue I thought I'd ask on the list >> before filing an issue. >> >> I created a Python2 virtual environment and installed IPython[notebook] >> using pip, plus scipy and some other stuff. I then opened an existing >> notebook, only to find on executing it that it claims scipy isn't installed. >> >> I noticed that the notebook had automatically opened with a Python 3 >> kernel, which I found very odd. sys.prefix tells me that it's running >> Python 3 from a completely unconnected virtual environment. >> >> Now I did, once upon a time, attempt to install a Python2 kernel, but my >> configuration control isn't good enough to remind me what I did. I can't >> trace any reference to the "unconnected virtual environment" anywhere in my >> ~/.ipython directory. This raises two questions: >> >> 1: Can anyone tell me where I may have installed this rogue kernel; and >> >> 2: It seems notebooks contain a kernel specification of sorts. Does this >> mean I can't write a notebook that will run in both Python 2 and Python 3? >> >> regards >> Steve >> -- >> Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 >> / @holdenweb >> >> >> >> >> >> >> -- >> Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 >> / @holdenweb >> >> >> >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Mon Jun 29 14:15:25 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 29 Jun 2015 11:15:25 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: I am +1 on having separate branches for ongoing 4.x work that include more than just bug fixes - especially for the notebook and widgets. I have a slight preference to have master always be the newer stuff, but don't feel too strongly about that. On Mon, Jun 29, 2015 at 10:54 AM, Jonathan Frederic wrote: >> I wanted to +1 the proposal to start creating branches for new versions >> when a feature freeze occurs. > > > +1 here too, I'm going to do that with ipywidgets for SciPy, create a 5.x > branch. There's a lot of stuff I want to get a jump start on, and SciPy is > a great time to do it. I don't want it to end up like numerous other > experiments, which end up getting thrown out and redone completely just > because of stagmentation and rebase difficulty. > > WRT to the documentation debt, I think it's important to remind everyone > that that is intentional! I've looked at adding JS docs a couple times now, > but when I brought it up we've decided as a group that it was lower priority > because we did not want to commit JavaScript APIs that we know will change. > > I think when we figure out how the front-end packaging and component > refactor will work, we definitely want to commit to **something**. > > On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay > wrote: >> >> I wanted to +1 the proposal to start creating branches for new versions >> when a feature freeze occurs. Independently of the discussion on phosphor, I >> completely agree with Min on the diagnosis that there is not enough >> available parallel work. >> >> Regarding phosphor and the work on refactoring the front-end, thanks for >> creating the centralized phosphor notebook repository in the organization. I >> did some experiments lately with the widgets and did not know where this >> could fall, or how to share it without requiring it to install phosphor etc. >> Coordination is also important for new developments, even when they have not >> yet achieved the stability of the main components of the project. >> >> Best, >> >> Sylvain >> >> >> >> On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout wrote: >>> >>> On 6/26/15 19:45, Fernando Perez wrote: >>> >>> While I hear very much the spirit of what you are saying, and I >>> certainly think that we can't lose sight that the *only* thing that >>> ultimately matters is whether we serve our users well or not, there's a >>> big piece that is already burning under us that probably can't wait. In >>> fact, at the last dev meeting, Jason already posted his new draft code >>> in this direction: >>> >>> https://github.com/jasongrout/phosphor-notebook >>> >>> >>> I just wanted to mention that I support what Fernando, Brian, and Chris >>> have said about moving forward with refactoring the notebook. We're making >>> good progress, even while still ramping up. For example, Steven Silvester >>> has put a lot of work recently in porting over the kernel javascript to >>> Typescript and phosphor (along with dependencies): >>> >>> https://github.com/jasongrout/phosphor-notebook/pull/2 >>> >>> I just put in an in-progress pull request for documenting the API for >>> kernels, kernelspecs, and sessions (which I realized when looking at the >>> kernel javascript file was woefully undocumented/incorrectly documented): >>> https://github.com/jupyter/notebook/pull/173. This shows our refactoring >>> work is also having an immediate direct impact on the current notebook as >>> well. >>> >>> In another message on this thread, Min suggested having a 5.x branch for >>> further development, like the phosphor notebook. For now, I think the >>> phosphor notebook can proceed as a separate project - it's totally a >>> front-end thing at this point, and we're doing enough code clean-up and >>> rewriting from js to typescript that I think it's all right to start in a >>> fresh repo. Which brings up another point: can we make an official Jupyter >>> repo for the phosphor notebook work, rather than using my personal repo? >>> I'm happy to continue hosting >>> https://github.com/jasongrout/phosphor-notebook/ in my personal github >>> account for the time being, or set up a temporary organization so we can >>> collaborate more effectively, but I think it would make more sense to bump >>> it up to an experimental repo in the jupyter github organization, developed >>> in parallel with the current notebook. >>> >>> Thomas, one thing to consider is that us working on a phosphor notebook >>> doesn't preclude interested people from enhancing the existing notebook in >>> the short term. We'd like the phosphor notebook to get to a comparable >>> state with the current notebook as quickly as possible, but it will still >>> take some time. >>> >>> Also, I totally agree with Thomas that dogfooding the notebook (and >>> watching/helping others actually use it to get work done) is *extremely* >>> important to understanding what we want here. And I also agree with others >>> on this thread that documentation is sorely lacking. We'll be working on >>> that in the phosphor notebook as we go along too. >>> >>> Thanks, >>> >>> Jason >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From benjaminrk at gmail.com Mon Jun 29 14:23:40 2015 From: benjaminrk at gmail.com (MinRK) Date: Mon, 29 Jun 2015 11:23:40 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: On Mon, Jun 29, 2015 at 11:15 AM, Brian Granger wrote: > I am +1 on having separate branches for ongoing 4.x work that include > more than just bug fixes - especially for the notebook and widgets. I > have a slight preference to have master always be the newer stuff, but > don't feel too strongly about that. > I think there's typically a transition point. For instance, we had long-running experiment/dev branches for kernel/qtconsole, notebook, and zmq-based IPython.parallel that weren't master. I think once these things become "good enough" they can hop over to master, and the minor-release branches can be bumped. We typically try to live in a "master always works" state, and some of these things can take a while to get there, in which case a feature branch is appropriate for the early stages of development. They can then become master once ready for "primetime", which is to say that we expect most IPython/Jupyter devs to be using it by default. But in general, I agree that master == newer stuff, as long as it's usable. -MinRK > > On Mon, Jun 29, 2015 at 10:54 AM, Jonathan Frederic > wrote: > >> I wanted to +1 the proposal to start creating branches for new versions > >> when a feature freeze occurs. > > > > > > +1 here too, I'm going to do that with ipywidgets for SciPy, create a 5.x > > branch. There's a lot of stuff I want to get a jump start on, and SciPy > is > > a great time to do it. I don't want it to end up like numerous other > > experiments, which end up getting thrown out and redone completely just > > because of stagmentation and rebase difficulty. > > > > WRT to the documentation debt, I think it's important to remind everyone > > that that is intentional! I've looked at adding JS docs a couple times > now, > > but when I brought it up we've decided as a group that it was lower > priority > > because we did not want to commit JavaScript APIs that we know will > change. > > > > I think when we figure out how the front-end packaging and component > > refactor will work, we definitely want to commit to **something**. > > > > On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay < > sylvain.corlay at gmail.com> > > wrote: > >> > >> I wanted to +1 the proposal to start creating branches for new versions > >> when a feature freeze occurs. Independently of the discussion on > phosphor, I > >> completely agree with Min on the diagnosis that there is not enough > >> available parallel work. > >> > >> Regarding phosphor and the work on refactoring the front-end, thanks for > >> creating the centralized phosphor notebook repository in the > organization. I > >> did some experiments lately with the widgets and did not know where this > >> could fall, or how to share it without requiring it to install phosphor > etc. > >> Coordination is also important for new developments, even when they > have not > >> yet achieved the stability of the main components of the project. > >> > >> Best, > >> > >> Sylvain > >> > >> > >> > >> On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout > wrote: > >>> > >>> On 6/26/15 19:45, Fernando Perez wrote: > >>> > >>> While I hear very much the spirit of what you are saying, and I > >>> certainly think that we can't lose sight that the *only* thing that > >>> ultimately matters is whether we serve our users well or not, there's a > >>> big piece that is already burning under us that probably can't wait. > In > >>> fact, at the last dev meeting, Jason already posted his new draft code > >>> in this direction: > >>> > >>> https://github.com/jasongrout/phosphor-notebook > >>> > >>> > >>> I just wanted to mention that I support what Fernando, Brian, and Chris > >>> have said about moving forward with refactoring the notebook. We're > making > >>> good progress, even while still ramping up. For example, Steven > Silvester > >>> has put a lot of work recently in porting over the kernel javascript to > >>> Typescript and phosphor (along with dependencies): > >>> > >>> https://github.com/jasongrout/phosphor-notebook/pull/2 > >>> > >>> I just put in an in-progress pull request for documenting the API for > >>> kernels, kernelspecs, and sessions (which I realized when looking at > the > >>> kernel javascript file was woefully undocumented/incorrectly > documented): > >>> https://github.com/jupyter/notebook/pull/173. This shows our > refactoring > >>> work is also having an immediate direct impact on the current notebook > as > >>> well. > >>> > >>> In another message on this thread, Min suggested having a 5.x branch > for > >>> further development, like the phosphor notebook. For now, I think the > >>> phosphor notebook can proceed as a separate project - it's totally a > >>> front-end thing at this point, and we're doing enough code clean-up and > >>> rewriting from js to typescript that I think it's all right to start > in a > >>> fresh repo. Which brings up another point: can we make an official > Jupyter > >>> repo for the phosphor notebook work, rather than using my personal > repo? > >>> I'm happy to continue hosting > >>> https://github.com/jasongrout/phosphor-notebook/ in my personal github > >>> account for the time being, or set up a temporary organization so we > can > >>> collaborate more effectively, but I think it would make more sense to > bump > >>> it up to an experimental repo in the jupyter github organization, > developed > >>> in parallel with the current notebook. > >>> > >>> Thomas, one thing to consider is that us working on a phosphor notebook > >>> doesn't preclude interested people from enhancing the existing > notebook in > >>> the short term. We'd like the phosphor notebook to get to a comparable > >>> state with the current notebook as quickly as possible, but it will > still > >>> take some time. > >>> > >>> Also, I totally agree with Thomas that dogfooding the notebook (and > >>> watching/helping others actually use it to get work done) is > *extremely* > >>> important to understanding what we want here. And I also agree with > others > >>> on this thread that documentation is sorely lacking. We'll be working > on > >>> that in the phosphor notebook as we go along too. > >>> > >>> Thanks, > >>> > >>> Jason > >>> > >>> _______________________________________________ > >>> IPython-dev mailing list > >>> IPython-dev at scipy.org > >>> http://mail.scipy.org/mailman/listinfo/ipython-dev > >>> > >> > >> > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Mon Jun 29 14:28:01 2015 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 29 Jun 2015 13:28:01 -0500 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: With the HubFlow (GitFlow) branch structure, I think this would look something like: - master (lastest tagged stable release) - develop (stable trunk) - feature - feature/phosphor (rebase -i develop) - hotfix/ (-> master, -> develop) - hotfix/docs - release/ cc'ing from http://westurner.org/tools/#hubflow .. index:: HubFlow .. _hubflow: HubFlow ~~~~~~~~~ | Src: https://github.com/datasift/gitflow | Docs: https://datasift.github.io/gitflow/ | Docs: https://datasift.github.io/gitflow/IntroducingGitFlow.html | Docs: https://datasift.github.io/gitflow/TheHubFlowTools.html HubFlow is a fork of GitFlow that adds extremely useful commands for working with Git and GitHub. HubFlow is a named branch workflow with mostly-automated merges between branches. Branch names are configurable; the defaults are as follows: +--------------------+-------------------------------------------------------------------------+ | **Branch Name** | **Description** | | | (and `Code Labels `__) | +--------------------+-------------------------------------------------------------------------+ | ``master`` | Stable trunk (latest release) | +--------------------+-------------------------------------------------------------------------+ | ``develop`` | Development main line | +--------------------+-------------------------------------------------------------------------+ | ``feature/`` | New features for the next release (e.g. ``ENH``, ``PRF``) | +--------------------+-------------------------------------------------------------------------+ | ``hotfix/`` | Fixes to merge to both ``master`` and ``develop`` | | | (e.g. ``BUG``, ``TST``, ``DOC``) | +--------------------+-------------------------------------------------------------------------+ | ``release/`` | In-progress release branches (e.g. ``RLS``) | +--------------------+-------------------------------------------------------------------------+ Creating a new release with :ref:`Git` and HubFlow: .. code:: bash git clone ssh://git at github.com/westurner/dotfiles # git checkout master git hf init ## Update versiontag in .git/config to prefix release tags with 'v' git config hubflow.prefix.versiontag=v #cat .git/config # ... # [hubflow "prefix"] # feature = feature/ # release = release/ # hotfix = hotfix/ # support = support/ # versiontag = v # git hf feature start ENH_print_hello_world ## commit, commit, commit git hf feature finish ENH_print_hello_world # ENH git hf release start 0.1.0 ## commit (e.g. update __version__, setup.py, release notes) git hf release finish 0.1.0 git hf release finish 0.1.0 git tag | grep 'v0.1.0' The GitFlow HubFlow illustrations are very helpful for visualizing and understanding any DVCS workflow: ``__. ****** with HTML formatting ***** git clone ssh://git at github.com/westurner/dotfiles # git checkout master git hf init ## Update versiontag in .git/config to prefix release tags with 'v' git config hubflow.prefix.versiontag=v #cat .git/config # ... # [hubflow "prefix"] # feature = feature/ # release = release/ # hotfix = hotfix/ # support = support/ # versiontag = v # git hf feature start ENH_print_hello_world ## commit, commit, commit git hf feature finish ENH_print_hello_world # ENH git hf release start 0.1.0 ## commit (e.g. update __version__, setup.py, release notes) git hf release finish 0.1.0 git hf release finish 0.1.0 git tag | grep 'v0.1.0' On Mon, Jun 29, 2015 at 1:15 PM, Brian Granger wrote: > I am +1 on having separate branches for ongoing 4.x work that include > more than just bug fixes - especially for the notebook and widgets. I > have a slight preference to have master always be the newer stuff, but > don't feel too strongly about that. > > On Mon, Jun 29, 2015 at 10:54 AM, Jonathan Frederic > wrote: > >> I wanted to +1 the proposal to start creating branches for new versions > >> when a feature freeze occurs. > > > > > > +1 here too, I'm going to do that with ipywidgets for SciPy, create a 5.x > > branch. There's a lot of stuff I want to get a jump start on, and SciPy > is > > a great time to do it. I don't want it to end up like numerous other > > experiments, which end up getting thrown out and redone completely just > > because of stagmentation and rebase difficulty. > > > > WRT to the documentation debt, I think it's important to remind everyone > > that that is intentional! I've looked at adding JS docs a couple times > now, > > but when I brought it up we've decided as a group that it was lower > priority > > because we did not want to commit JavaScript APIs that we know will > change. > > > > I think when we figure out how the front-end packaging and component > > refactor will work, we definitely want to commit to **something**. > > > > On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay < > sylvain.corlay at gmail.com> > > wrote: > >> > >> I wanted to +1 the proposal to start creating branches for new versions > >> when a feature freeze occurs. Independently of the discussion on > phosphor, I > >> completely agree with Min on the diagnosis that there is not enough > >> available parallel work. > >> > >> Regarding phosphor and the work on refactoring the front-end, thanks for > >> creating the centralized phosphor notebook repository in the > organization. I > >> did some experiments lately with the widgets and did not know where this > >> could fall, or how to share it without requiring it to install phosphor > etc. > >> Coordination is also important for new developments, even when they > have not > >> yet achieved the stability of the main components of the project. > >> > >> Best, > >> > >> Sylvain > >> > >> > >> > >> On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout > wrote: > >>> > >>> On 6/26/15 19:45, Fernando Perez wrote: > >>> > >>> While I hear very much the spirit of what you are saying, and I > >>> certainly think that we can't lose sight that the *only* thing that > >>> ultimately matters is whether we serve our users well or not, there's a > >>> big piece that is already burning under us that probably can't wait. > In > >>> fact, at the last dev meeting, Jason already posted his new draft code > >>> in this direction: > >>> > >>> https://github.com/jasongrout/phosphor-notebook > >>> > >>> > >>> I just wanted to mention that I support what Fernando, Brian, and Chris > >>> have said about moving forward with refactoring the notebook. We're > making > >>> good progress, even while still ramping up. For example, Steven > Silvester > >>> has put a lot of work recently in porting over the kernel javascript to > >>> Typescript and phosphor (along with dependencies): > >>> > >>> https://github.com/jasongrout/phosphor-notebook/pull/2 > >>> > >>> I just put in an in-progress pull request for documenting the API for > >>> kernels, kernelspecs, and sessions (which I realized when looking at > the > >>> kernel javascript file was woefully undocumented/incorrectly > documented): > >>> https://github.com/jupyter/notebook/pull/173. This shows our > refactoring > >>> work is also having an immediate direct impact on the current notebook > as > >>> well. > >>> > >>> In another message on this thread, Min suggested having a 5.x branch > for > >>> further development, like the phosphor notebook. For now, I think the > >>> phosphor notebook can proceed as a separate project - it's totally a > >>> front-end thing at this point, and we're doing enough code clean-up and > >>> rewriting from js to typescript that I think it's all right to start > in a > >>> fresh repo. Which brings up another point: can we make an official > Jupyter > >>> repo for the phosphor notebook work, rather than using my personal > repo? > >>> I'm happy to continue hosting > >>> https://github.com/jasongrout/phosphor-notebook/ in my personal github > >>> account for the time being, or set up a temporary organization so we > can > >>> collaborate more effectively, but I think it would make more sense to > bump > >>> it up to an experimental repo in the jupyter github organization, > developed > >>> in parallel with the current notebook. > >>> > >>> Thomas, one thing to consider is that us working on a phosphor notebook > >>> doesn't preclude interested people from enhancing the existing > notebook in > >>> the short term. We'd like the phosphor notebook to get to a comparable > >>> state with the current notebook as quickly as possible, but it will > still > >>> take some time. > >>> > >>> Also, I totally agree with Thomas that dogfooding the notebook (and > >>> watching/helping others actually use it to get work done) is > *extremely* > >>> important to understanding what we want here. And I also agree with > others > >>> on this thread that documentation is sorely lacking. We'll be working > on > >>> that in the phosphor notebook as we go along too. > >>> > >>> Thanks, > >>> > >>> Jason > >>> > >>> _______________________________________________ > >>> IPython-dev mailing list > >>> IPython-dev at scipy.org > >>> http://mail.scipy.org/mailman/listinfo/ipython-dev > >>> > >> > >> > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Mon Jun 29 14:33:56 2015 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 29 Jun 2015 13:33:56 -0500 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: On Mon, Jun 29, 2015 at 1:23 PM, MinRK wrote: > > > On Mon, Jun 29, 2015 at 11:15 AM, Brian Granger > wrote: > >> I am +1 on having separate branches for ongoing 4.x work that include >> more than just bug fixes - especially for the notebook and widgets. I >> have a slight preference to have master always be the newer stuff, but >> don't feel too strongly about that. >> > > I think there's typically a transition point. For instance, we had > long-running experiment/dev branches for kernel/qtconsole, notebook, and > zmq-based IPython.parallel that weren't master. I think once these things > become "good enough" they can hop over to master, and the minor-release > branches can be bumped. We typically try to live in a "master always works" > state, and some of these things can take a while to get there, in which > case a feature branch is appropriate for the early stages of development. > They can then become master once ready for "primetime", which is to say > that we expect most IPython/Jupyter devs to be using it by default. But in > general, I agree that master == newer stuff, as long as it's usable. > I think there's a really strong argument for having stable master, tests passing on develop always, and separate branches. As a novice, when I 'git clone https://github.com/ipython/ipython', I sort of expect a passing build (preferably the latest tagged release; as with, again, sorry, HubFlow). With HubFlow, branch names are configurable: This is from a HubFlow .git/config: [hubflow "branch"] master = master develop = develop [hubflow "prefix"] feature = feature/ release = release/ hotfix = hotfix/ support = support/ versiontag = v > > -MinRK > > >> >> On Mon, Jun 29, 2015 at 10:54 AM, Jonathan Frederic >> wrote: >> >> I wanted to +1 the proposal to start creating branches for new versions >> >> when a feature freeze occurs. >> > >> > >> > +1 here too, I'm going to do that with ipywidgets for SciPy, create a >> 5.x >> > branch. There's a lot of stuff I want to get a jump start on, and >> SciPy is >> > a great time to do it. I don't want it to end up like numerous other >> > experiments, which end up getting thrown out and redone completely just >> > because of stagmentation and rebase difficulty. >> > >> > WRT to the documentation debt, I think it's important to remind everyone >> > that that is intentional! I've looked at adding JS docs a couple times >> now, >> > but when I brought it up we've decided as a group that it was lower >> priority >> > because we did not want to commit JavaScript APIs that we know will >> change. >> > >> > I think when we figure out how the front-end packaging and component >> > refactor will work, we definitely want to commit to **something**. >> > >> > On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay < >> sylvain.corlay at gmail.com> >> > wrote: >> >> >> >> I wanted to +1 the proposal to start creating branches for new versions >> >> when a feature freeze occurs. Independently of the discussion on >> phosphor, I >> >> completely agree with Min on the diagnosis that there is not enough >> >> available parallel work. >> >> >> >> Regarding phosphor and the work on refactoring the front-end, thanks >> for >> >> creating the centralized phosphor notebook repository in the >> organization. I >> >> did some experiments lately with the widgets and did not know where >> this >> >> could fall, or how to share it without requiring it to install >> phosphor etc. >> >> Coordination is also important for new developments, even when they >> have not >> >> yet achieved the stability of the main components of the project. >> >> >> >> Best, >> >> >> >> Sylvain >> >> >> >> >> >> >> >> On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout >> wrote: >> >>> >> >>> On 6/26/15 19:45, Fernando Perez wrote: >> >>> >> >>> While I hear very much the spirit of what you are saying, and I >> >>> certainly think that we can't lose sight that the *only* thing that >> >>> ultimately matters is whether we serve our users well or not, there's >> a >> >>> big piece that is already burning under us that probably can't wait. >> In >> >>> fact, at the last dev meeting, Jason already posted his new draft code >> >>> in this direction: >> >>> >> >>> https://github.com/jasongrout/phosphor-notebook >> >>> >> >>> >> >>> I just wanted to mention that I support what Fernando, Brian, and >> Chris >> >>> have said about moving forward with refactoring the notebook. We're >> making >> >>> good progress, even while still ramping up. For example, Steven >> Silvester >> >>> has put a lot of work recently in porting over the kernel javascript >> to >> >>> Typescript and phosphor (along with dependencies): >> >>> >> >>> https://github.com/jasongrout/phosphor-notebook/pull/2 >> >>> >> >>> I just put in an in-progress pull request for documenting the API for >> >>> kernels, kernelspecs, and sessions (which I realized when looking at >> the >> >>> kernel javascript file was woefully undocumented/incorrectly >> documented): >> >>> https://github.com/jupyter/notebook/pull/173. This shows our >> refactoring >> >>> work is also having an immediate direct impact on the current >> notebook as >> >>> well. >> >>> >> >>> In another message on this thread, Min suggested having a 5.x branch >> for >> >>> further development, like the phosphor notebook. For now, I think the >> >>> phosphor notebook can proceed as a separate project - it's totally a >> >>> front-end thing at this point, and we're doing enough code clean-up >> and >> >>> rewriting from js to typescript that I think it's all right to start >> in a >> >>> fresh repo. Which brings up another point: can we make an official >> Jupyter >> >>> repo for the phosphor notebook work, rather than using my personal >> repo? >> >>> I'm happy to continue hosting >> >>> https://github.com/jasongrout/phosphor-notebook/ in my personal >> github >> >>> account for the time being, or set up a temporary organization so we >> can >> >>> collaborate more effectively, but I think it would make more sense to >> bump >> >>> it up to an experimental repo in the jupyter github organization, >> developed >> >>> in parallel with the current notebook. >> >>> >> >>> Thomas, one thing to consider is that us working on a phosphor >> notebook >> >>> doesn't preclude interested people from enhancing the existing >> notebook in >> >>> the short term. We'd like the phosphor notebook to get to a >> comparable >> >>> state with the current notebook as quickly as possible, but it will >> still >> >>> take some time. >> >>> >> >>> Also, I totally agree with Thomas that dogfooding the notebook (and >> >>> watching/helping others actually use it to get work done) is >> *extremely* >> >>> important to understanding what we want here. And I also agree with >> others >> >>> on this thread that documentation is sorely lacking. We'll be >> working on >> >>> that in the phosphor notebook as we go along too. >> >>> >> >>> Thanks, >> >>> >> >>> Jason >> >>> >> >>> _______________________________________________ >> >>> IPython-dev mailing list >> >>> IPython-dev at scipy.org >> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >>> >> >> >> >> >> >> _______________________________________________ >> >> IPython-dev mailing list >> >> IPython-dev at scipy.org >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> > >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> >> >> >> -- >> Brian E. Granger >> Cal Poly State University, San Luis Obispo >> @ellisonbg on Twitter and GitHub >> bgranger at calpoly.edu and ellisonbg at gmail.com >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Mon Jun 29 14:35:18 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 29 Jun 2015 11:35:18 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: Good points - I guess with the magnitude of the changes that are going to be happening with the notebook and widget it would be nice to have a working master. +1 then to the 5.0 brance idea for those repos. On Mon, Jun 29, 2015 at 11:33 AM, Wes Turner wrote: > > > On Mon, Jun 29, 2015 at 1:23 PM, MinRK wrote: >> >> >> >> On Mon, Jun 29, 2015 at 11:15 AM, Brian Granger >> wrote: >>> >>> I am +1 on having separate branches for ongoing 4.x work that include >>> more than just bug fixes - especially for the notebook and widgets. I >>> have a slight preference to have master always be the newer stuff, but >>> don't feel too strongly about that. >> >> >> I think there's typically a transition point. For instance, we had >> long-running experiment/dev branches for kernel/qtconsole, notebook, and >> zmq-based IPython.parallel that weren't master. I think once these things >> become "good enough" they can hop over to master, and the minor-release >> branches can be bumped. We typically try to live in a "master always works" >> state, and some of these things can take a while to get there, in which case >> a feature branch is appropriate for the early stages of development. They >> can then become master once ready for "primetime", which is to say that we >> expect most IPython/Jupyter devs to be using it by default. But in general, >> I agree that master == newer stuff, as long as it's usable. > > > I think there's a really strong argument for having stable master, tests > passing on develop always, > and separate branches. > > As a novice, when I 'git clone https://github.com/ipython/ipython', > I sort of expect a passing build > (preferably the latest tagged release; as with, again, sorry, HubFlow). > > With HubFlow, branch names are configurable: This is from a HubFlow > .git/config: > > [hubflow "branch"] > master = master > develop = develop > > [hubflow "prefix"] > feature = feature/ > release = release/ > hotfix = hotfix/ > support = support/ > versiontag = v > >> >> >> -MinRK >> >>> >>> >>> On Mon, Jun 29, 2015 at 10:54 AM, Jonathan Frederic >>> wrote: >>> >> I wanted to +1 the proposal to start creating branches for new >>> >> versions >>> >> when a feature freeze occurs. >>> > >>> > >>> > +1 here too, I'm going to do that with ipywidgets for SciPy, create a >>> > 5.x >>> > branch. There's a lot of stuff I want to get a jump start on, and >>> > SciPy is >>> > a great time to do it. I don't want it to end up like numerous other >>> > experiments, which end up getting thrown out and redone completely just >>> > because of stagmentation and rebase difficulty. >>> > >>> > WRT to the documentation debt, I think it's important to remind >>> > everyone >>> > that that is intentional! I've looked at adding JS docs a couple times >>> > now, >>> > but when I brought it up we've decided as a group that it was lower >>> > priority >>> > because we did not want to commit JavaScript APIs that we know will >>> > change. >>> > >>> > I think when we figure out how the front-end packaging and component >>> > refactor will work, we definitely want to commit to **something**. >>> > >>> > On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay >>> > >>> > wrote: >>> >> >>> >> I wanted to +1 the proposal to start creating branches for new >>> >> versions >>> >> when a feature freeze occurs. Independently of the discussion on >>> >> phosphor, I >>> >> completely agree with Min on the diagnosis that there is not enough >>> >> available parallel work. >>> >> >>> >> Regarding phosphor and the work on refactoring the front-end, thanks >>> >> for >>> >> creating the centralized phosphor notebook repository in the >>> >> organization. I >>> >> did some experiments lately with the widgets and did not know where >>> >> this >>> >> could fall, or how to share it without requiring it to install >>> >> phosphor etc. >>> >> Coordination is also important for new developments, even when they >>> >> have not >>> >> yet achieved the stability of the main components of the project. >>> >> >>> >> Best, >>> >> >>> >> Sylvain >>> >> >>> >> >>> >> >>> >> On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout >>> >> wrote: >>> >>> >>> >>> On 6/26/15 19:45, Fernando Perez wrote: >>> >>> >>> >>> While I hear very much the spirit of what you are saying, and I >>> >>> certainly think that we can't lose sight that the *only* thing that >>> >>> ultimately matters is whether we serve our users well or not, there's >>> >>> a >>> >>> big piece that is already burning under us that probably can't wait. >>> >>> In >>> >>> fact, at the last dev meeting, Jason already posted his new draft >>> >>> code >>> >>> in this direction: >>> >>> >>> >>> https://github.com/jasongrout/phosphor-notebook >>> >>> >>> >>> >>> >>> I just wanted to mention that I support what Fernando, Brian, and >>> >>> Chris >>> >>> have said about moving forward with refactoring the notebook. We're >>> >>> making >>> >>> good progress, even while still ramping up. For example, Steven >>> >>> Silvester >>> >>> has put a lot of work recently in porting over the kernel javascript >>> >>> to >>> >>> Typescript and phosphor (along with dependencies): >>> >>> >>> >>> https://github.com/jasongrout/phosphor-notebook/pull/2 >>> >>> >>> >>> I just put in an in-progress pull request for documenting the API for >>> >>> kernels, kernelspecs, and sessions (which I realized when looking at >>> >>> the >>> >>> kernel javascript file was woefully undocumented/incorrectly >>> >>> documented): >>> >>> https://github.com/jupyter/notebook/pull/173. This shows our >>> >>> refactoring >>> >>> work is also having an immediate direct impact on the current >>> >>> notebook as >>> >>> well. >>> >>> >>> >>> In another message on this thread, Min suggested having a 5.x branch >>> >>> for >>> >>> further development, like the phosphor notebook. For now, I think >>> >>> the >>> >>> phosphor notebook can proceed as a separate project - it's totally a >>> >>> front-end thing at this point, and we're doing enough code clean-up >>> >>> and >>> >>> rewriting from js to typescript that I think it's all right to start >>> >>> in a >>> >>> fresh repo. Which brings up another point: can we make an official >>> >>> Jupyter >>> >>> repo for the phosphor notebook work, rather than using my personal >>> >>> repo? >>> >>> I'm happy to continue hosting >>> >>> https://github.com/jasongrout/phosphor-notebook/ in my personal >>> >>> github >>> >>> account for the time being, or set up a temporary organization so we >>> >>> can >>> >>> collaborate more effectively, but I think it would make more sense to >>> >>> bump >>> >>> it up to an experimental repo in the jupyter github organization, >>> >>> developed >>> >>> in parallel with the current notebook. >>> >>> >>> >>> Thomas, one thing to consider is that us working on a phosphor >>> >>> notebook >>> >>> doesn't preclude interested people from enhancing the existing >>> >>> notebook in >>> >>> the short term. We'd like the phosphor notebook to get to a >>> >>> comparable >>> >>> state with the current notebook as quickly as possible, but it will >>> >>> still >>> >>> take some time. >>> >>> >>> >>> Also, I totally agree with Thomas that dogfooding the notebook (and >>> >>> watching/helping others actually use it to get work done) is >>> >>> *extremely* >>> >>> important to understanding what we want here. And I also agree with >>> >>> others >>> >>> on this thread that documentation is sorely lacking. We'll be >>> >>> working on >>> >>> that in the phosphor notebook as we go along too. >>> >>> >>> >>> Thanks, >>> >>> >>> >>> Jason >>> >>> >>> >>> _______________________________________________ >>> >>> IPython-dev mailing list >>> >>> IPython-dev at scipy.org >>> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >>> >> >>> >> >>> >> _______________________________________________ >>> >> IPython-dev mailing list >>> >> IPython-dev at scipy.org >>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> >>> > >>> > >>> > _______________________________________________ >>> > IPython-dev mailing list >>> > IPython-dev at scipy.org >>> > http://mail.scipy.org/mailman/listinfo/ipython-dev >>> > >>> >>> >>> >>> -- >>> Brian E. Granger >>> Cal Poly State University, San Luis Obispo >>> @ellisonbg on Twitter and GitHub >>> bgranger at calpoly.edu and ellisonbg at gmail.com >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From benjaminrk at gmail.com Mon Jun 29 14:36:52 2015 From: benjaminrk at gmail.com (MinRK) Date: Mon, 29 Jun 2015 11:36:52 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: Thanks, we are familiar with GitFlow, and have decided against using it. We find it doesn't reflect our relationship to branches and feature work. Working with PRs directly against master is ~always what we want, and it is only for unusually long-running changes that we have a need for a feature branch, and there is no distinction between develop and master. -MinRK On Mon, Jun 29, 2015 at 11:28 AM, Wes Turner wrote: > With the HubFlow (GitFlow) branch structure, I think this would look > something like: > > - master (lastest tagged stable release) > - develop (stable trunk) > - feature > - feature/phosphor (rebase -i develop) > - hotfix/ (-> master, -> develop) > - hotfix/docs > - release/ > > > cc'ing from http://westurner.org/tools/#hubflow > > .. index:: HubFlow > .. _hubflow: > > HubFlow > ~~~~~~~~~ > | Src: https://github.com/datasift/gitflow > | Docs: https://datasift.github.io/gitflow/ > | Docs: https://datasift.github.io/gitflow/IntroducingGitFlow.html > | Docs: https://datasift.github.io/gitflow/TheHubFlowTools.html > > HubFlow is a fork of GitFlow > that adds extremely useful commands for working with Git and GitHub. > > HubFlow is a named branch workflow with mostly-automated merges > between branches. > > Branch names are configurable; the defaults are as follows: > > > +--------------------+-------------------------------------------------------------------------+ > | **Branch Name** | **Description** | > | | (and `Code Labels `__) | > +--------------------+-------------------------------------------------------------------------+ > | ``master`` | Stable trunk (latest release) | > +--------------------+-------------------------------------------------------------------------+ > | ``develop`` | Development main line | > +--------------------+-------------------------------------------------------------------------+ > | ``feature/`` | New features for the next release (e.g. ``ENH``, ``PRF``) | > +--------------------+-------------------------------------------------------------------------+ > | ``hotfix/`` | Fixes to merge to both ``master`` and ``develop`` | > | | (e.g. ``BUG``, ``TST``, ``DOC``) | > +--------------------+-------------------------------------------------------------------------+ > | ``release/`` | In-progress release branches (e.g. ``RLS``) | > +--------------------+-------------------------------------------------------------------------+ > > Creating a new release with :ref:`Git` and HubFlow: > > .. code:: bash > > git clone ssh://git at github.com/westurner/dotfiles > # git checkout master > git hf init > ## Update versiontag in .git/config to prefix release tags with 'v' > git config hubflow.prefix.versiontag=v > #cat .git/config # ... > # [hubflow "prefix"] > # feature = feature/ > # release = release/ > # hotfix = hotfix/ > # support = support/ > # versiontag = v > # > git hf feature start ENH_print_hello_world > ## commit, commit, commit > git hf feature finish ENH_print_hello_world # ENH > git hf release start 0.1.0 > ## commit (e.g. update __version__, setup.py, release notes) > git hf release finish 0.1.0 > git hf release finish 0.1.0 > git tag | grep 'v0.1.0' > > The GitFlow HubFlow illustrations are very helpful for visualizing > and understanding any DVCS workflow: > ``__. > > > ****** with HTML formatting ***** > > > git clone ssh://git at github.com/westurner/dotfiles > # git checkout master > git hf init > ## Update versiontag in .git/config to prefix release tags with 'v' > git config hubflow.prefix.versiontag=v > #cat .git/config # ... > # [hubflow "prefix"] > # feature = feature/ > # release = release/ > # hotfix = hotfix/ > # support = support/ > # versiontag = v > # > git hf feature start ENH_print_hello_world > ## commit, commit, commit > git hf feature finish ENH_print_hello_world # ENH > git hf release start 0.1.0 > ## commit (e.g. update __version__, setup.py, release notes) > git hf release finish 0.1.0 > git hf release finish 0.1.0 > git tag | grep 'v0.1.0' > > > On Mon, Jun 29, 2015 at 1:15 PM, Brian Granger > wrote: > >> I am +1 on having separate branches for ongoing 4.x work that include >> more than just bug fixes - especially for the notebook and widgets. I >> have a slight preference to have master always be the newer stuff, but >> don't feel too strongly about that. >> >> On Mon, Jun 29, 2015 at 10:54 AM, Jonathan Frederic >> wrote: >> >> I wanted to +1 the proposal to start creating branches for new versions >> >> when a feature freeze occurs. >> > >> > >> > +1 here too, I'm going to do that with ipywidgets for SciPy, create a >> 5.x >> > branch. There's a lot of stuff I want to get a jump start on, and >> SciPy is >> > a great time to do it. I don't want it to end up like numerous other >> > experiments, which end up getting thrown out and redone completely just >> > because of stagmentation and rebase difficulty. >> > >> > WRT to the documentation debt, I think it's important to remind everyone >> > that that is intentional! I've looked at adding JS docs a couple times >> now, >> > but when I brought it up we've decided as a group that it was lower >> priority >> > because we did not want to commit JavaScript APIs that we know will >> change. >> > >> > I think when we figure out how the front-end packaging and component >> > refactor will work, we definitely want to commit to **something**. >> > >> > On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay < >> sylvain.corlay at gmail.com> >> > wrote: >> >> >> >> I wanted to +1 the proposal to start creating branches for new versions >> >> when a feature freeze occurs. Independently of the discussion on >> phosphor, I >> >> completely agree with Min on the diagnosis that there is not enough >> >> available parallel work. >> >> >> >> Regarding phosphor and the work on refactoring the front-end, thanks >> for >> >> creating the centralized phosphor notebook repository in the >> organization. I >> >> did some experiments lately with the widgets and did not know where >> this >> >> could fall, or how to share it without requiring it to install >> phosphor etc. >> >> Coordination is also important for new developments, even when they >> have not >> >> yet achieved the stability of the main components of the project. >> >> >> >> Best, >> >> >> >> Sylvain >> >> >> >> >> >> >> >> On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout >> wrote: >> >>> >> >>> On 6/26/15 19:45, Fernando Perez wrote: >> >>> >> >>> While I hear very much the spirit of what you are saying, and I >> >>> certainly think that we can't lose sight that the *only* thing that >> >>> ultimately matters is whether we serve our users well or not, there's >> a >> >>> big piece that is already burning under us that probably can't wait. >> In >> >>> fact, at the last dev meeting, Jason already posted his new draft code >> >>> in this direction: >> >>> >> >>> https://github.com/jasongrout/phosphor-notebook >> >>> >> >>> >> >>> I just wanted to mention that I support what Fernando, Brian, and >> Chris >> >>> have said about moving forward with refactoring the notebook. We're >> making >> >>> good progress, even while still ramping up. For example, Steven >> Silvester >> >>> has put a lot of work recently in porting over the kernel javascript >> to >> >>> Typescript and phosphor (along with dependencies): >> >>> >> >>> https://github.com/jasongrout/phosphor-notebook/pull/2 >> >>> >> >>> I just put in an in-progress pull request for documenting the API for >> >>> kernels, kernelspecs, and sessions (which I realized when looking at >> the >> >>> kernel javascript file was woefully undocumented/incorrectly >> documented): >> >>> https://github.com/jupyter/notebook/pull/173. This shows our >> refactoring >> >>> work is also having an immediate direct impact on the current >> notebook as >> >>> well. >> >>> >> >>> In another message on this thread, Min suggested having a 5.x branch >> for >> >>> further development, like the phosphor notebook. For now, I think the >> >>> phosphor notebook can proceed as a separate project - it's totally a >> >>> front-end thing at this point, and we're doing enough code clean-up >> and >> >>> rewriting from js to typescript that I think it's all right to start >> in a >> >>> fresh repo. Which brings up another point: can we make an official >> Jupyter >> >>> repo for the phosphor notebook work, rather than using my personal >> repo? >> >>> I'm happy to continue hosting >> >>> https://github.com/jasongrout/phosphor-notebook/ in my personal >> github >> >>> account for the time being, or set up a temporary organization so we >> can >> >>> collaborate more effectively, but I think it would make more sense to >> bump >> >>> it up to an experimental repo in the jupyter github organization, >> developed >> >>> in parallel with the current notebook. >> >>> >> >>> Thomas, one thing to consider is that us working on a phosphor >> notebook >> >>> doesn't preclude interested people from enhancing the existing >> notebook in >> >>> the short term. We'd like the phosphor notebook to get to a >> comparable >> >>> state with the current notebook as quickly as possible, but it will >> still >> >>> take some time. >> >>> >> >>> Also, I totally agree with Thomas that dogfooding the notebook (and >> >>> watching/helping others actually use it to get work done) is >> *extremely* >> >>> important to understanding what we want here. And I also agree with >> others >> >>> on this thread that documentation is sorely lacking. We'll be >> working on >> >>> that in the phosphor notebook as we go along too. >> >>> >> >>> Thanks, >> >>> >> >>> Jason >> >>> >> >>> _______________________________________________ >> >>> IPython-dev mailing list >> >>> IPython-dev at scipy.org >> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >>> >> >> >> >> >> >> _______________________________________________ >> >> IPython-dev mailing list >> >> IPython-dev at scipy.org >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> > >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> >> >> >> -- >> Brian E. Granger >> Cal Poly State University, San Luis Obispo >> @ellisonbg on Twitter and GitHub >> bgranger at calpoly.edu and ellisonbg at gmail.com >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Mon Jun 29 14:44:09 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 29 Jun 2015 11:44:09 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: Fernando, Could you start the documentation stuff as a separate thread? This one is getting a bit difficult to follow (remembering why email is so painful :( Cheers, Brian On Sat, Jun 27, 2015 at 3:53 PM, Fernando Perez wrote: > Here's an email that Andrew sent me privately, but with his authorization, > I'm replying to it on the public list as I think his concerns have a lot of > merit and I'm sure others would be interested in the topic as well... > > > On Fri, Jun 26, 2015 at 9:56 PM, Andrew Gibiansky > wrote: >> >> Fernando, >> >> However, I would like to urge you and the IPython team to consider >> focusing a little bit of effort towards something that has gone unmentioned: >> documentation. >> >> The IPython documentation has suffered from a few main problems: >> >> It is often out of date. The messaging spec that was accessible, as of a >> few weeks ago, was out of date and simply wrong about some things in IPython >> 3.1. >> Significant portions are undocumented, or documented only through their >> original IPEPs. For example, the backbone widgets! I am currently working >> with a student for GSOC to implement widget support for IHaskell, and he >> regularly has to resort to spelunking through the IPython source to >> implement the same functionality, rather than looking at a spec. >> Finally, and most importantly, the frontend is completely undocumented. As >> far as I can tell, the suggested way towards developing anything for the >> frontend is to grep through the source code, for example for which events >> are available to be bound to. The result of this is that developing frontend >> custom.js or extensions is nigh impossible; even simple ones become a >> gargantuan task requiring you to delve deep into IPython JS source code, and >> more extensive extensions can only be done by core IPython contributors (or >> people with similarly deep knowledge). As someone who is very familiar with >> IPython (having written IHaskell and a few frontend extensions and >> customizations), it is very difficult even for me to get anything done on >> the frontend, much less someone with less experience. I believe this >> significantly hinders development of new extensions, widgets, and frontends. >> >> Seeing Thomas' suggestion, I think it would be really great if the team >> could tackle the occasionally poor documentation right now. (I say >> 'occasionally' because the documentation is still much much better than many >> other projects!) From an external standpoint, it seems like the amount of >> technical debt in IPython is growing somewhat unmanageable to deal with for >> an outsider, in part because of the lack of documentation. Even if now is >> not the right time for a focus on documentation, it would be wonderful if >> this were brought up and if perhaps it were scoped out as part of the >> schedule for the upcoming months. >> >> Again, please take this in the best way possible ? this is not meant as >> criticism of the way IPython is being developed but rather as an expression >> of what would be important to me, a very frequent user of IPython and >> developer of a fairly popular backend. With all of this in mind, I want to >> say that I'm incredibly impressed with the quality and pace of development, >> and it's exciting to see plans for 4.0 and 5.0 being made! > > > These concerns are absolutely spot-on, and we're painfully aware of the > impact that the rather sad state of our docs has on users and developers of > the system alike. Furthermore, we know that it creates a significant > barrier to entry to new contributions, and to growing a healthier community > of participation around the project. > > We have, sadly, dug a pretty deep hole here, and it will take some real > effort to climb back out of it, so this isn't going to happen overnight. > > We've been, over the last few months, working very hard on trying to secure > some resources to ensure that the core team can remain employed, which is an > even more critical concern for some of us, as I hope you'll understand :) > > We hope to have some good news to report on that front before too long, and > once we can get some proper resources in place, we actually will put > documentation efforts as a first-class priority. Even before that, we have > a few small steps in the right direction: > > - Min went this spring to the WriteTheDocs conference, to start meeting > technical writers and get a bit more acquainted with that community, > > - Brian has already hired some new summer students who are starting to > improve the layout of the websites. That's not the same as writing a bunch > of new docs, but having better websites will help. The PRs are here: > https://github.com/jupyter/design/issues/15, > https://github.com/jupyter/design/issues/16, help always welcome. > > - We've actually already started work on restructuring all the repos for > better docs, and I know that Thomas, Brian, Jess, Jon, Matthias and others > (sorry folks for whoever I'm not crediting directly) have put work into it. > At least now we have an overall skeleton so that the navigation of the docs > for the overall project is more comprehensible: > > http://jupyter.readthedocs.org/en/latest/ > > and we're also working on tooling to smooth the integration of notebooks > into the sphinx workflow. > > > These efforts are all partial, and cooking at slow simmer because people are > more or less at max capacity trying to get 4.0 out the door and with the > Scipy conference breathing down our necks. > > I would say though that, once 4.0 is out the door, with this new skeleton in > place, contributing docs will be a place where the community can > *definitely* help the project. I know until now it was hard to even help > with documentation, because the layout of our docs was so messy and > monolithic. We're trying to precisely help with making it easier to > *contribute*, please let us know if this new structure looks like a step in > the right direction or not... > > > But we are not, not at all, ignoring your concerns. Hopefully in a few > days/weeks we'll have a bit more we can share on that front that will help > too... > > Cheers, > > f > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From wes.turner at gmail.com Mon Jun 29 14:45:12 2015 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 29 Jun 2015 13:45:12 -0500 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: On Mon, Jun 29, 2015 at 1:36 PM, MinRK wrote: > Thanks, we are familiar with GitFlow, and have decided against using it. > We find it doesn't reflect our relationship to branches and feature work. > Working with PRs directly against master is ~always what we want, and it is > only for unusually long-running changes that we have a need for a feature > branch, and there is no distinction between develop and master. > Thanks for the feedback. * [ ] Which development branch should I fork and PR from? (master) So, each feature/ and hotfix/ branch is a PR against "develop =" (or, master). * [ ] Something like this probably works: [hubflow "branch"] master = stable develop = master [hubflow "prefix"] feature = feature/ release = release/ hotfix = hotfix/ support = support/ versiontag = v That is all. > -MinRK > > On Mon, Jun 29, 2015 at 11:28 AM, Wes Turner wrote: > >> With the HubFlow (GitFlow) branch structure, I think this would look >> something like: >> >> - master (lastest tagged stable release) >> - develop (stable trunk) >> - feature >> - feature/phosphor (rebase -i develop) >> - hotfix/ (-> master, -> develop) >> - hotfix/docs >> - release/ >> >> >> cc'ing from http://westurner.org/tools/#hubflow >> >> .. index:: HubFlow >> .. _hubflow: >> >> HubFlow >> ~~~~~~~~~ >> | Src: https://github.com/datasift/gitflow >> | Docs: https://datasift.github.io/gitflow/ >> | Docs: https://datasift.github.io/gitflow/IntroducingGitFlow.html >> | Docs: https://datasift.github.io/gitflow/TheHubFlowTools.html >> >> HubFlow is a fork of GitFlow >> that adds extremely useful commands for working with Git and GitHub. >> >> HubFlow is a named branch workflow with mostly-automated merges >> between branches. >> >> Branch names are configurable; the defaults are as follows: >> >> >> +--------------------+-------------------------------------------------------------------------+ >> | **Branch Name** | **Description** | >> | | (and `Code Labels `__) | >> +--------------------+-------------------------------------------------------------------------+ >> | ``master`` | Stable trunk (latest release) | >> +--------------------+-------------------------------------------------------------------------+ >> | ``develop`` | Development main line | >> +--------------------+-------------------------------------------------------------------------+ >> | ``feature/`` | New features for the next release (e.g. ``ENH``, ``PRF``) | >> +--------------------+-------------------------------------------------------------------------+ >> | ``hotfix/`` | Fixes to merge to both ``master`` and ``develop`` | >> | | (e.g. ``BUG``, ``TST``, ``DOC``) | >> +--------------------+-------------------------------------------------------------------------+ >> | ``release/`` | In-progress release branches (e.g. ``RLS``) | >> +--------------------+-------------------------------------------------------------------------+ >> >> Creating a new release with :ref:`Git` and HubFlow: >> >> .. code:: bash >> >> git clone ssh://git at github.com/westurner/dotfiles >> # git checkout master >> git hf init >> ## Update versiontag in .git/config to prefix release tags with 'v' >> git config hubflow.prefix.versiontag=v >> #cat .git/config # ... >> # [hubflow "prefix"] >> # feature = feature/ >> # release = release/ >> # hotfix = hotfix/ >> # support = support/ >> # versiontag = v >> # >> git hf feature start ENH_print_hello_world >> ## commit, commit, commit >> git hf feature finish ENH_print_hello_world # ENH >> git hf release start 0.1.0 >> ## commit (e.g. update __version__, setup.py, release notes) >> git hf release finish 0.1.0 >> git hf release finish 0.1.0 >> git tag | grep 'v0.1.0' >> >> The GitFlow HubFlow illustrations are very helpful for visualizing >> and understanding any DVCS workflow: >> ``__. >> >> >> ****** with HTML formatting ***** >> >> >> git clone ssh://git at github.com/westurner/dotfiles >> # git checkout master >> git hf init >> ## Update versiontag in .git/config to prefix release tags with 'v' >> git config hubflow.prefix.versiontag=v >> #cat .git/config # ... >> # [hubflow "prefix"] >> # feature = feature/ >> # release = release/ >> # hotfix = hotfix/ >> # support = support/ >> # versiontag = v >> # >> git hf feature start ENH_print_hello_world >> ## commit, commit, commit >> git hf feature finish ENH_print_hello_world # ENH >> git hf release start 0.1.0 >> ## commit (e.g. update __version__, setup.py, release notes) >> git hf release finish 0.1.0 >> git hf release finish 0.1.0 >> git tag | grep 'v0.1.0' >> >> >> On Mon, Jun 29, 2015 at 1:15 PM, Brian Granger >> wrote: >> >>> I am +1 on having separate branches for ongoing 4.x work that include >>> more than just bug fixes - especially for the notebook and widgets. I >>> have a slight preference to have master always be the newer stuff, but >>> don't feel too strongly about that. >>> >>> On Mon, Jun 29, 2015 at 10:54 AM, Jonathan Frederic >>> wrote: >>> >> I wanted to +1 the proposal to start creating branches for new >>> versions >>> >> when a feature freeze occurs. >>> > >>> > >>> > +1 here too, I'm going to do that with ipywidgets for SciPy, create a >>> 5.x >>> > branch. There's a lot of stuff I want to get a jump start on, and >>> SciPy is >>> > a great time to do it. I don't want it to end up like numerous other >>> > experiments, which end up getting thrown out and redone completely just >>> > because of stagmentation and rebase difficulty. >>> > >>> > WRT to the documentation debt, I think it's important to remind >>> everyone >>> > that that is intentional! I've looked at adding JS docs a couple >>> times now, >>> > but when I brought it up we've decided as a group that it was lower >>> priority >>> > because we did not want to commit JavaScript APIs that we know will >>> change. >>> > >>> > I think when we figure out how the front-end packaging and component >>> > refactor will work, we definitely want to commit to **something**. >>> > >>> > On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay < >>> sylvain.corlay at gmail.com> >>> > wrote: >>> >> >>> >> I wanted to +1 the proposal to start creating branches for new >>> versions >>> >> when a feature freeze occurs. Independently of the discussion on >>> phosphor, I >>> >> completely agree with Min on the diagnosis that there is not enough >>> >> available parallel work. >>> >> >>> >> Regarding phosphor and the work on refactoring the front-end, thanks >>> for >>> >> creating the centralized phosphor notebook repository in the >>> organization. I >>> >> did some experiments lately with the widgets and did not know where >>> this >>> >> could fall, or how to share it without requiring it to install >>> phosphor etc. >>> >> Coordination is also important for new developments, even when they >>> have not >>> >> yet achieved the stability of the main components of the project. >>> >> >>> >> Best, >>> >> >>> >> Sylvain >>> >> >>> >> >>> >> >>> >> On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout >>> wrote: >>> >>> >>> >>> On 6/26/15 19:45, Fernando Perez wrote: >>> >>> >>> >>> While I hear very much the spirit of what you are saying, and I >>> >>> certainly think that we can't lose sight that the *only* thing that >>> >>> ultimately matters is whether we serve our users well or not, >>> there's a >>> >>> big piece that is already burning under us that probably can't >>> wait. In >>> >>> fact, at the last dev meeting, Jason already posted his new draft >>> code >>> >>> in this direction: >>> >>> >>> >>> https://github.com/jasongrout/phosphor-notebook >>> >>> >>> >>> >>> >>> I just wanted to mention that I support what Fernando, Brian, and >>> Chris >>> >>> have said about moving forward with refactoring the notebook. We're >>> making >>> >>> good progress, even while still ramping up. For example, Steven >>> Silvester >>> >>> has put a lot of work recently in porting over the kernel javascript >>> to >>> >>> Typescript and phosphor (along with dependencies): >>> >>> >>> >>> https://github.com/jasongrout/phosphor-notebook/pull/2 >>> >>> >>> >>> I just put in an in-progress pull request for documenting the API for >>> >>> kernels, kernelspecs, and sessions (which I realized when looking at >>> the >>> >>> kernel javascript file was woefully undocumented/incorrectly >>> documented): >>> >>> https://github.com/jupyter/notebook/pull/173. This shows our >>> refactoring >>> >>> work is also having an immediate direct impact on the current >>> notebook as >>> >>> well. >>> >>> >>> >>> In another message on this thread, Min suggested having a 5.x branch >>> for >>> >>> further development, like the phosphor notebook. For now, I think >>> the >>> >>> phosphor notebook can proceed as a separate project - it's totally a >>> >>> front-end thing at this point, and we're doing enough code clean-up >>> and >>> >>> rewriting from js to typescript that I think it's all right to start >>> in a >>> >>> fresh repo. Which brings up another point: can we make an official >>> Jupyter >>> >>> repo for the phosphor notebook work, rather than using my personal >>> repo? >>> >>> I'm happy to continue hosting >>> >>> https://github.com/jasongrout/phosphor-notebook/ in my personal >>> github >>> >>> account for the time being, or set up a temporary organization so we >>> can >>> >>> collaborate more effectively, but I think it would make more sense >>> to bump >>> >>> it up to an experimental repo in the jupyter github organization, >>> developed >>> >>> in parallel with the current notebook. >>> >>> >>> >>> Thomas, one thing to consider is that us working on a phosphor >>> notebook >>> >>> doesn't preclude interested people from enhancing the existing >>> notebook in >>> >>> the short term. We'd like the phosphor notebook to get to a >>> comparable >>> >>> state with the current notebook as quickly as possible, but it will >>> still >>> >>> take some time. >>> >>> >>> >>> Also, I totally agree with Thomas that dogfooding the notebook (and >>> >>> watching/helping others actually use it to get work done) is >>> *extremely* >>> >>> important to understanding what we want here. And I also agree with >>> others >>> >>> on this thread that documentation is sorely lacking. We'll be >>> working on >>> >>> that in the phosphor notebook as we go along too. >>> >>> >>> >>> Thanks, >>> >>> >>> >>> Jason >>> >>> >>> >>> _______________________________________________ >>> >>> IPython-dev mailing list >>> >>> IPython-dev at scipy.org >>> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >>> >> >>> >> >>> >> _______________________________________________ >>> >> IPython-dev mailing list >>> >> IPython-dev at scipy.org >>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> >>> > >>> > >>> > _______________________________________________ >>> > IPython-dev mailing list >>> > IPython-dev at scipy.org >>> > http://mail.scipy.org/mailman/listinfo/ipython-dev >>> > >>> >>> >>> >>> -- >>> Brian E. Granger >>> Cal Poly State University, San Luis Obispo >>> @ellisonbg on Twitter and GitHub >>> bgranger at calpoly.edu and ellisonbg at gmail.com >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Mon Jun 29 14:56:17 2015 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 29 Jun 2015 13:56:17 -0500 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: And one last thing, So, without a "stable" branch (e.g. "master ="), all of the CI build scripts are pinned to each of the separate repositories? Use Case: build latest everything from source. On Mon, Jun 29, 2015 at 1:45 PM, Wes Turner wrote: > > > On Mon, Jun 29, 2015 at 1:36 PM, MinRK wrote: > >> Thanks, we are familiar with GitFlow, and have decided against using it. >> We find it doesn't reflect our relationship to branches and feature work. >> Working with PRs directly against master is ~always what we want, and it is >> only for unusually long-running changes that we have a need for a feature >> branch, and there is no distinction between develop and master. >> > > > Thanks for the feedback. > > * [ ] Which development branch should I fork and PR from? (master) > > So, each feature/ and hotfix/ branch is a PR against "develop =" (or, > master). > > * [ ] Something like this probably works: > > [hubflow "branch"] > master = stable > develop = master > > [hubflow "prefix"] > feature = feature/ > release = release/ > hotfix = hotfix/ > support = support/ > versiontag = v > > That is all. > > > > > >> -MinRK >> >> On Mon, Jun 29, 2015 at 11:28 AM, Wes Turner >> wrote: >> >>> With the HubFlow (GitFlow) branch structure, I think this would look >>> something like: >>> >>> - master (lastest tagged stable release) >>> - develop (stable trunk) >>> - feature >>> - feature/phosphor (rebase -i develop) >>> - hotfix/ (-> master, -> develop) >>> - hotfix/docs >>> - release/ >>> >>> >>> cc'ing from http://westurner.org/tools/#hubflow >>> >>> .. index:: HubFlow >>> .. _hubflow: >>> >>> HubFlow >>> ~~~~~~~~~ >>> | Src: https://github.com/datasift/gitflow >>> | Docs: https://datasift.github.io/gitflow/ >>> | Docs: https://datasift.github.io/gitflow/IntroducingGitFlow.html >>> | Docs: https://datasift.github.io/gitflow/TheHubFlowTools.html >>> >>> HubFlow is a fork of GitFlow >>> that adds extremely useful commands for working with Git and GitHub. >>> >>> HubFlow is a named branch workflow with mostly-automated merges >>> between branches. >>> >>> Branch names are configurable; the defaults are as follows: >>> >>> >>> +--------------------+-------------------------------------------------------------------------+ >>> | **Branch Name** | **Description** | >>> | | (and `Code Labels `__) | >>> +--------------------+-------------------------------------------------------------------------+ >>> | ``master`` | Stable trunk (latest release) | >>> +--------------------+-------------------------------------------------------------------------+ >>> | ``develop`` | Development main line | >>> +--------------------+-------------------------------------------------------------------------+ >>> | ``feature/`` | New features for the next release (e.g. ``ENH``, ``PRF``) | >>> +--------------------+-------------------------------------------------------------------------+ >>> | ``hotfix/`` | Fixes to merge to both ``master`` and ``develop`` | >>> | | (e.g. ``BUG``, ``TST``, ``DOC``) | >>> +--------------------+-------------------------------------------------------------------------+ >>> | ``release/`` | In-progress release branches (e.g. ``RLS``) | >>> +--------------------+-------------------------------------------------------------------------+ >>> >>> Creating a new release with :ref:`Git` and HubFlow: >>> >>> .. code:: bash >>> >>> git clone ssh://git at github.com/westurner/dotfiles >>> # git checkout master >>> git hf init >>> ## Update versiontag in .git/config to prefix release tags with 'v' >>> git config hubflow.prefix.versiontag=v >>> #cat .git/config # ... >>> # [hubflow "prefix"] >>> # feature = feature/ >>> # release = release/ >>> # hotfix = hotfix/ >>> # support = support/ >>> # versiontag = v >>> # >>> git hf feature start ENH_print_hello_world >>> ## commit, commit, commit >>> git hf feature finish ENH_print_hello_world # ENH >>> git hf release start 0.1.0 >>> ## commit (e.g. update __version__, setup.py, release notes) >>> git hf release finish 0.1.0 >>> git hf release finish 0.1.0 >>> git tag | grep 'v0.1.0' >>> >>> The GitFlow HubFlow illustrations are very helpful for visualizing >>> and understanding any DVCS workflow: >>> ``__. >>> >>> >>> ****** with HTML formatting ***** >>> >>> >>> git clone ssh://git at github.com/westurner/dotfiles >>> # git checkout master >>> git hf init >>> ## Update versiontag in .git/config to prefix release tags with 'v' >>> git config hubflow.prefix.versiontag=v >>> #cat .git/config # ... >>> # [hubflow "prefix"] >>> # feature = feature/ >>> # release = release/ >>> # hotfix = hotfix/ >>> # support = support/ >>> # versiontag = v >>> # >>> git hf feature start ENH_print_hello_world >>> ## commit, commit, commit >>> git hf feature finish ENH_print_hello_world # ENH >>> git hf release start 0.1.0 >>> ## commit (e.g. update __version__, setup.py, release notes) >>> git hf release finish 0.1.0 >>> git hf release finish 0.1.0 >>> git tag | grep 'v0.1.0' >>> >>> >>> On Mon, Jun 29, 2015 at 1:15 PM, Brian Granger >>> wrote: >>> >>>> I am +1 on having separate branches for ongoing 4.x work that include >>>> more than just bug fixes - especially for the notebook and widgets. I >>>> have a slight preference to have master always be the newer stuff, but >>>> don't feel too strongly about that. >>>> >>>> On Mon, Jun 29, 2015 at 10:54 AM, Jonathan Frederic >>>> wrote: >>>> >> I wanted to +1 the proposal to start creating branches for new >>>> versions >>>> >> when a feature freeze occurs. >>>> > >>>> > >>>> > +1 here too, I'm going to do that with ipywidgets for SciPy, create a >>>> 5.x >>>> > branch. There's a lot of stuff I want to get a jump start on, and >>>> SciPy is >>>> > a great time to do it. I don't want it to end up like numerous other >>>> > experiments, which end up getting thrown out and redone completely >>>> just >>>> > because of stagmentation and rebase difficulty. >>>> > >>>> > WRT to the documentation debt, I think it's important to remind >>>> everyone >>>> > that that is intentional! I've looked at adding JS docs a couple >>>> times now, >>>> > but when I brought it up we've decided as a group that it was lower >>>> priority >>>> > because we did not want to commit JavaScript APIs that we know will >>>> change. >>>> > >>>> > I think when we figure out how the front-end packaging and component >>>> > refactor will work, we definitely want to commit to **something**. >>>> > >>>> > On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay < >>>> sylvain.corlay at gmail.com> >>>> > wrote: >>>> >> >>>> >> I wanted to +1 the proposal to start creating branches for new >>>> versions >>>> >> when a feature freeze occurs. Independently of the discussion on >>>> phosphor, I >>>> >> completely agree with Min on the diagnosis that there is not enough >>>> >> available parallel work. >>>> >> >>>> >> Regarding phosphor and the work on refactoring the front-end, thanks >>>> for >>>> >> creating the centralized phosphor notebook repository in the >>>> organization. I >>>> >> did some experiments lately with the widgets and did not know where >>>> this >>>> >> could fall, or how to share it without requiring it to install >>>> phosphor etc. >>>> >> Coordination is also important for new developments, even when they >>>> have not >>>> >> yet achieved the stability of the main components of the project. >>>> >> >>>> >> Best, >>>> >> >>>> >> Sylvain >>>> >> >>>> >> >>>> >> >>>> >> On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout >>>> wrote: >>>> >>> >>>> >>> On 6/26/15 19:45, Fernando Perez wrote: >>>> >>> >>>> >>> While I hear very much the spirit of what you are saying, and I >>>> >>> certainly think that we can't lose sight that the *only* thing that >>>> >>> ultimately matters is whether we serve our users well or not, >>>> there's a >>>> >>> big piece that is already burning under us that probably can't >>>> wait. In >>>> >>> fact, at the last dev meeting, Jason already posted his new draft >>>> code >>>> >>> in this direction: >>>> >>> >>>> >>> https://github.com/jasongrout/phosphor-notebook >>>> >>> >>>> >>> >>>> >>> I just wanted to mention that I support what Fernando, Brian, and >>>> Chris >>>> >>> have said about moving forward with refactoring the notebook. >>>> We're making >>>> >>> good progress, even while still ramping up. For example, Steven >>>> Silvester >>>> >>> has put a lot of work recently in porting over the kernel >>>> javascript to >>>> >>> Typescript and phosphor (along with dependencies): >>>> >>> >>>> >>> https://github.com/jasongrout/phosphor-notebook/pull/2 >>>> >>> >>>> >>> I just put in an in-progress pull request for documenting the API >>>> for >>>> >>> kernels, kernelspecs, and sessions (which I realized when looking >>>> at the >>>> >>> kernel javascript file was woefully undocumented/incorrectly >>>> documented): >>>> >>> https://github.com/jupyter/notebook/pull/173. This shows our >>>> refactoring >>>> >>> work is also having an immediate direct impact on the current >>>> notebook as >>>> >>> well. >>>> >>> >>>> >>> In another message on this thread, Min suggested having a 5.x >>>> branch for >>>> >>> further development, like the phosphor notebook. For now, I think >>>> the >>>> >>> phosphor notebook can proceed as a separate project - it's totally a >>>> >>> front-end thing at this point, and we're doing enough code clean-up >>>> and >>>> >>> rewriting from js to typescript that I think it's all right to >>>> start in a >>>> >>> fresh repo. Which brings up another point: can we make an >>>> official Jupyter >>>> >>> repo for the phosphor notebook work, rather than using my personal >>>> repo? >>>> >>> I'm happy to continue hosting >>>> >>> https://github.com/jasongrout/phosphor-notebook/ in my personal >>>> github >>>> >>> account for the time being, or set up a temporary organization so >>>> we can >>>> >>> collaborate more effectively, but I think it would make more sense >>>> to bump >>>> >>> it up to an experimental repo in the jupyter github organization, >>>> developed >>>> >>> in parallel with the current notebook. >>>> >>> >>>> >>> Thomas, one thing to consider is that us working on a phosphor >>>> notebook >>>> >>> doesn't preclude interested people from enhancing the existing >>>> notebook in >>>> >>> the short term. We'd like the phosphor notebook to get to a >>>> comparable >>>> >>> state with the current notebook as quickly as possible, but it will >>>> still >>>> >>> take some time. >>>> >>> >>>> >>> Also, I totally agree with Thomas that dogfooding the notebook (and >>>> >>> watching/helping others actually use it to get work done) is >>>> *extremely* >>>> >>> important to understanding what we want here. And I also agree >>>> with others >>>> >>> on this thread that documentation is sorely lacking. We'll be >>>> working on >>>> >>> that in the phosphor notebook as we go along too. >>>> >>> >>>> >>> Thanks, >>>> >>> >>>> >>> Jason >>>> >>> >>>> >>> _______________________________________________ >>>> >>> IPython-dev mailing list >>>> >>> IPython-dev at scipy.org >>>> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>> >>>> >> >>>> >> >>>> >> _______________________________________________ >>>> >> IPython-dev mailing list >>>> >> IPython-dev at scipy.org >>>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >> >>>> > >>>> > >>>> > _______________________________________________ >>>> > IPython-dev mailing list >>>> > IPython-dev at scipy.org >>>> > http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> > >>>> >>>> >>>> >>>> -- >>>> Brian E. Granger >>>> Cal Poly State University, San Luis Obispo >>>> @ellisonbg on Twitter and GitHub >>>> bgranger at calpoly.edu and ellisonbg at gmail.com >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Jun 29 14:57:47 2015 From: benjaminrk at gmail.com (MinRK) Date: Mon, 29 Jun 2015 11:57:47 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: On Mon, Jun 29, 2015 at 11:56 AM, Wes Turner wrote: > And one last thing, > > So, without a "stable" branch (e.g. "master ="), > all of the CI build scripts > are pinned to each of the separate repositories? > > Use Case: build latest everything from source. > Latest everything = master > > > On Mon, Jun 29, 2015 at 1:45 PM, Wes Turner wrote: > >> >> >> On Mon, Jun 29, 2015 at 1:36 PM, MinRK wrote: >> >>> Thanks, we are familiar with GitFlow, and have decided against using it. >>> We find it doesn't reflect our relationship to branches and feature work. >>> Working with PRs directly against master is ~always what we want, and it is >>> only for unusually long-running changes that we have a need for a feature >>> branch, and there is no distinction between develop and master. >>> >> >> >> Thanks for the feedback. >> >> * [ ] Which development branch should I fork and PR from? (master) >> >> So, each feature/ and hotfix/ branch is a PR against "develop =" (or, >> master). >> >> * [ ] Something like this probably works: >> >> [hubflow "branch"] >> master = stable >> develop = master >> >> [hubflow "prefix"] >> feature = feature/ >> release = release/ >> hotfix = hotfix/ >> support = support/ >> versiontag = v >> >> That is all. >> >> >> >> >> >>> -MinRK >>> >>> On Mon, Jun 29, 2015 at 11:28 AM, Wes Turner >>> wrote: >>> >>>> With the HubFlow (GitFlow) branch structure, I think this would look >>>> something like: >>>> >>>> - master (lastest tagged stable release) >>>> - develop (stable trunk) >>>> - feature >>>> - feature/phosphor (rebase -i develop) >>>> - hotfix/ (-> master, -> develop) >>>> - hotfix/docs >>>> - release/ >>>> >>>> >>>> cc'ing from http://westurner.org/tools/#hubflow >>>> >>>> .. index:: HubFlow >>>> .. _hubflow: >>>> >>>> HubFlow >>>> ~~~~~~~~~ >>>> | Src: https://github.com/datasift/gitflow >>>> | Docs: https://datasift.github.io/gitflow/ >>>> | Docs: https://datasift.github.io/gitflow/IntroducingGitFlow.html >>>> | Docs: https://datasift.github.io/gitflow/TheHubFlowTools.html >>>> >>>> HubFlow is a fork of GitFlow >>>> that adds extremely useful commands for working with Git and GitHub. >>>> >>>> HubFlow is a named branch workflow with mostly-automated merges >>>> between branches. >>>> >>>> Branch names are configurable; the defaults are as follows: >>>> >>>> >>>> +--------------------+-------------------------------------------------------------------------+ >>>> | **Branch Name** | **Description** | >>>> | | (and `Code Labels `__) | >>>> +--------------------+-------------------------------------------------------------------------+ >>>> | ``master`` | Stable trunk (latest release) | >>>> +--------------------+-------------------------------------------------------------------------+ >>>> | ``develop`` | Development main line | >>>> +--------------------+-------------------------------------------------------------------------+ >>>> | ``feature/`` | New features for the next release (e.g. ``ENH``, ``PRF``) | >>>> +--------------------+-------------------------------------------------------------------------+ >>>> | ``hotfix/`` | Fixes to merge to both ``master`` and ``develop`` | >>>> | | (e.g. ``BUG``, ``TST``, ``DOC``) | >>>> +--------------------+-------------------------------------------------------------------------+ >>>> | ``release/`` | In-progress release branches (e.g. ``RLS``) | >>>> +--------------------+-------------------------------------------------------------------------+ >>>> >>>> Creating a new release with :ref:`Git` and HubFlow: >>>> >>>> .. code:: bash >>>> >>>> git clone ssh://git at github.com/westurner/dotfiles >>>> # git checkout master >>>> git hf init >>>> ## Update versiontag in .git/config to prefix release tags with 'v' >>>> git config hubflow.prefix.versiontag=v >>>> #cat .git/config # ... >>>> # [hubflow "prefix"] >>>> # feature = feature/ >>>> # release = release/ >>>> # hotfix = hotfix/ >>>> # support = support/ >>>> # versiontag = v >>>> # >>>> git hf feature start ENH_print_hello_world >>>> ## commit, commit, commit >>>> git hf feature finish ENH_print_hello_world # ENH >>>> git hf release start 0.1.0 >>>> ## commit (e.g. update __version__, setup.py, release notes) >>>> git hf release finish 0.1.0 >>>> git hf release finish 0.1.0 >>>> git tag | grep 'v0.1.0' >>>> >>>> The GitFlow HubFlow illustrations are very helpful for visualizing >>>> and understanding any DVCS workflow: >>>> ``__. >>>> >>>> >>>> ****** with HTML formatting ***** >>>> >>>> >>>> git clone ssh://git at github.com/westurner/dotfiles >>>> # git checkout master >>>> git hf init >>>> ## Update versiontag in .git/config to prefix release tags with 'v' >>>> git config hubflow.prefix.versiontag=v >>>> #cat .git/config # ... >>>> # [hubflow "prefix"] >>>> # feature = feature/ >>>> # release = release/ >>>> # hotfix = hotfix/ >>>> # support = support/ >>>> # versiontag = v >>>> # >>>> git hf feature start ENH_print_hello_world >>>> ## commit, commit, commit >>>> git hf feature finish ENH_print_hello_world # ENH >>>> git hf release start 0.1.0 >>>> ## commit (e.g. update __version__, setup.py, release notes) >>>> git hf release finish 0.1.0 >>>> git hf release finish 0.1.0 >>>> git tag | grep 'v0.1.0' >>>> >>>> >>>> On Mon, Jun 29, 2015 at 1:15 PM, Brian Granger >>>> wrote: >>>> >>>>> I am +1 on having separate branches for ongoing 4.x work that include >>>>> more than just bug fixes - especially for the notebook and widgets. I >>>>> have a slight preference to have master always be the newer stuff, but >>>>> don't feel too strongly about that. >>>>> >>>>> On Mon, Jun 29, 2015 at 10:54 AM, Jonathan Frederic >>>>> wrote: >>>>> >> I wanted to +1 the proposal to start creating branches for new >>>>> versions >>>>> >> when a feature freeze occurs. >>>>> > >>>>> > >>>>> > +1 here too, I'm going to do that with ipywidgets for SciPy, create >>>>> a 5.x >>>>> > branch. There's a lot of stuff I want to get a jump start on, and >>>>> SciPy is >>>>> > a great time to do it. I don't want it to end up like numerous other >>>>> > experiments, which end up getting thrown out and redone completely >>>>> just >>>>> > because of stagmentation and rebase difficulty. >>>>> > >>>>> > WRT to the documentation debt, I think it's important to remind >>>>> everyone >>>>> > that that is intentional! I've looked at adding JS docs a couple >>>>> times now, >>>>> > but when I brought it up we've decided as a group that it was lower >>>>> priority >>>>> > because we did not want to commit JavaScript APIs that we know will >>>>> change. >>>>> > >>>>> > I think when we figure out how the front-end packaging and component >>>>> > refactor will work, we definitely want to commit to **something**. >>>>> > >>>>> > On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay < >>>>> sylvain.corlay at gmail.com> >>>>> > wrote: >>>>> >> >>>>> >> I wanted to +1 the proposal to start creating branches for new >>>>> versions >>>>> >> when a feature freeze occurs. Independently of the discussion on >>>>> phosphor, I >>>>> >> completely agree with Min on the diagnosis that there is not enough >>>>> >> available parallel work. >>>>> >> >>>>> >> Regarding phosphor and the work on refactoring the front-end, >>>>> thanks for >>>>> >> creating the centralized phosphor notebook repository in the >>>>> organization. I >>>>> >> did some experiments lately with the widgets and did not know where >>>>> this >>>>> >> could fall, or how to share it without requiring it to install >>>>> phosphor etc. >>>>> >> Coordination is also important for new developments, even when they >>>>> have not >>>>> >> yet achieved the stability of the main components of the project. >>>>> >> >>>>> >> Best, >>>>> >> >>>>> >> Sylvain >>>>> >> >>>>> >> >>>>> >> >>>>> >> On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout >>>>> wrote: >>>>> >>> >>>>> >>> On 6/26/15 19:45, Fernando Perez wrote: >>>>> >>> >>>>> >>> While I hear very much the spirit of what you are saying, and I >>>>> >>> certainly think that we can't lose sight that the *only* thing that >>>>> >>> ultimately matters is whether we serve our users well or not, >>>>> there's a >>>>> >>> big piece that is already burning under us that probably can't >>>>> wait. In >>>>> >>> fact, at the last dev meeting, Jason already posted his new draft >>>>> code >>>>> >>> in this direction: >>>>> >>> >>>>> >>> https://github.com/jasongrout/phosphor-notebook >>>>> >>> >>>>> >>> >>>>> >>> I just wanted to mention that I support what Fernando, Brian, and >>>>> Chris >>>>> >>> have said about moving forward with refactoring the notebook. >>>>> We're making >>>>> >>> good progress, even while still ramping up. For example, Steven >>>>> Silvester >>>>> >>> has put a lot of work recently in porting over the kernel >>>>> javascript to >>>>> >>> Typescript and phosphor (along with dependencies): >>>>> >>> >>>>> >>> https://github.com/jasongrout/phosphor-notebook/pull/2 >>>>> >>> >>>>> >>> I just put in an in-progress pull request for documenting the API >>>>> for >>>>> >>> kernels, kernelspecs, and sessions (which I realized when looking >>>>> at the >>>>> >>> kernel javascript file was woefully undocumented/incorrectly >>>>> documented): >>>>> >>> https://github.com/jupyter/notebook/pull/173. This shows our >>>>> refactoring >>>>> >>> work is also having an immediate direct impact on the current >>>>> notebook as >>>>> >>> well. >>>>> >>> >>>>> >>> In another message on this thread, Min suggested having a 5.x >>>>> branch for >>>>> >>> further development, like the phosphor notebook. For now, I think >>>>> the >>>>> >>> phosphor notebook can proceed as a separate project - it's totally >>>>> a >>>>> >>> front-end thing at this point, and we're doing enough code >>>>> clean-up and >>>>> >>> rewriting from js to typescript that I think it's all right to >>>>> start in a >>>>> >>> fresh repo. Which brings up another point: can we make an >>>>> official Jupyter >>>>> >>> repo for the phosphor notebook work, rather than using my personal >>>>> repo? >>>>> >>> I'm happy to continue hosting >>>>> >>> https://github.com/jasongrout/phosphor-notebook/ in my personal >>>>> github >>>>> >>> account for the time being, or set up a temporary organization so >>>>> we can >>>>> >>> collaborate more effectively, but I think it would make more sense >>>>> to bump >>>>> >>> it up to an experimental repo in the jupyter github organization, >>>>> developed >>>>> >>> in parallel with the current notebook. >>>>> >>> >>>>> >>> Thomas, one thing to consider is that us working on a phosphor >>>>> notebook >>>>> >>> doesn't preclude interested people from enhancing the existing >>>>> notebook in >>>>> >>> the short term. We'd like the phosphor notebook to get to a >>>>> comparable >>>>> >>> state with the current notebook as quickly as possible, but it >>>>> will still >>>>> >>> take some time. >>>>> >>> >>>>> >>> Also, I totally agree with Thomas that dogfooding the notebook (and >>>>> >>> watching/helping others actually use it to get work done) is >>>>> *extremely* >>>>> >>> important to understanding what we want here. And I also agree >>>>> with others >>>>> >>> on this thread that documentation is sorely lacking. We'll be >>>>> working on >>>>> >>> that in the phosphor notebook as we go along too. >>>>> >>> >>>>> >>> Thanks, >>>>> >>> >>>>> >>> Jason >>>>> >>> >>>>> >>> _______________________________________________ >>>>> >>> IPython-dev mailing list >>>>> >>> IPython-dev at scipy.org >>>>> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>> >>> >>>>> >> >>>>> >> >>>>> >> _______________________________________________ >>>>> >> IPython-dev mailing list >>>>> >> IPython-dev at scipy.org >>>>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>> >> >>>>> > >>>>> > >>>>> > _______________________________________________ >>>>> > IPython-dev mailing list >>>>> > IPython-dev at scipy.org >>>>> > http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>> > >>>>> >>>>> >>>>> >>>>> -- >>>>> Brian E. Granger >>>>> Cal Poly State University, San Luis Obispo >>>>> @ellisonbg on Twitter and GitHub >>>>> bgranger at calpoly.edu and ellisonbg at gmail.com >>>>> _______________________________________________ >>>>> IPython-dev mailing list >>>>> IPython-dev at scipy.org >>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>> >>>> >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Mon Jun 29 15:01:52 2015 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 29 Jun 2015 14:01:52 -0500 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: <5591474B.3060506@jasongrout.org> Message-ID: Got it. Thanks again! To clarify, Use Case: build latest stable release everything from source (e.g. a production stack) Here's theses: https://github.com/ipython/ipython/wiki/Dev:-Index https://github.com/ipython/ipython/wiki/Dev:-GitHub-workflow https://github.com/ipython/ipython/wiki/Dev:-Coding-style On Mon, Jun 29, 2015 at 1:57 PM, MinRK wrote: > > > On Mon, Jun 29, 2015 at 11:56 AM, Wes Turner wrote: > >> And one last thing, >> >> So, without a "stable" branch (e.g. "master ="), >> all of the CI build scripts >> are pinned to each of the separate repositories? >> >> Use Case: build latest everything from source. >> > > Latest everything = master > > >> >> >> On Mon, Jun 29, 2015 at 1:45 PM, Wes Turner wrote: >> >>> >>> >>> On Mon, Jun 29, 2015 at 1:36 PM, MinRK wrote: >>> >>>> Thanks, we are familiar with GitFlow, and have decided against using >>>> it. We find it doesn't reflect our relationship to branches and feature >>>> work. Working with PRs directly against master is ~always what we want, and >>>> it is only for unusually long-running changes that we have a need for a >>>> feature branch, and there is no distinction between develop and master. >>>> >>> >>> >>> Thanks for the feedback. >>> >>> * [ ] Which development branch should I fork and PR from? (master) >>> >>> So, each feature/ and hotfix/ branch is a PR against "develop =" (or, >>> master). >>> >>> * [ ] Something like this probably works: >>> >>> [hubflow "branch"] >>> master = stable >>> develop = master >>> >>> [hubflow "prefix"] >>> feature = feature/ >>> release = release/ >>> hotfix = hotfix/ >>> support = support/ >>> versiontag = v >>> >>> That is all. >>> >>> >>> >>> >>> >>>> -MinRK >>>> >>>> On Mon, Jun 29, 2015 at 11:28 AM, Wes Turner >>>> wrote: >>>> >>>>> With the HubFlow (GitFlow) branch structure, I think this would look >>>>> something like: >>>>> >>>>> - master (lastest tagged stable release) >>>>> - develop (stable trunk) >>>>> - feature >>>>> - feature/phosphor (rebase -i develop) >>>>> - hotfix/ (-> master, -> develop) >>>>> - hotfix/docs >>>>> - release/ >>>>> >>>>> >>>>> cc'ing from http://westurner.org/tools/#hubflow >>>>> >>>>> .. index:: HubFlow >>>>> .. _hubflow: >>>>> >>>>> HubFlow >>>>> ~~~~~~~~~ >>>>> | Src: https://github.com/datasift/gitflow >>>>> | Docs: https://datasift.github.io/gitflow/ >>>>> | Docs: https://datasift.github.io/gitflow/IntroducingGitFlow.html >>>>> | Docs: https://datasift.github.io/gitflow/TheHubFlowTools.html >>>>> >>>>> HubFlow is a fork of GitFlow >>>>> that adds extremely useful commands for working with Git and GitHub. >>>>> >>>>> HubFlow is a named branch workflow with mostly-automated merges >>>>> between branches. >>>>> >>>>> Branch names are configurable; the defaults are as follows: >>>>> >>>>> >>>>> +--------------------+-------------------------------------------------------------------------+ >>>>> | **Branch Name** | **Description** | >>>>> | | (and `Code Labels `__) | >>>>> +--------------------+-------------------------------------------------------------------------+ >>>>> | ``master`` | Stable trunk (latest release) | >>>>> +--------------------+-------------------------------------------------------------------------+ >>>>> | ``develop`` | Development main line | >>>>> +--------------------+-------------------------------------------------------------------------+ >>>>> | ``feature/`` | New features for the next release (e.g. ``ENH``, ``PRF``) | >>>>> +--------------------+-------------------------------------------------------------------------+ >>>>> | ``hotfix/`` | Fixes to merge to both ``master`` and ``develop`` | >>>>> | | (e.g. ``BUG``, ``TST``, ``DOC``) | >>>>> +--------------------+-------------------------------------------------------------------------+ >>>>> | ``release/`` | In-progress release branches (e.g. ``RLS``) | >>>>> +--------------------+-------------------------------------------------------------------------+ >>>>> >>>>> Creating a new release with :ref:`Git` and HubFlow: >>>>> >>>>> .. code:: bash >>>>> >>>>> git clone ssh://git at github.com/westurner/dotfiles >>>>> # git checkout master >>>>> git hf init >>>>> ## Update versiontag in .git/config to prefix release tags with 'v' >>>>> git config hubflow.prefix.versiontag=v >>>>> #cat .git/config # ... >>>>> # [hubflow "prefix"] >>>>> # feature = feature/ >>>>> # release = release/ >>>>> # hotfix = hotfix/ >>>>> # support = support/ >>>>> # versiontag = v >>>>> # >>>>> git hf feature start ENH_print_hello_world >>>>> ## commit, commit, commit >>>>> git hf feature finish ENH_print_hello_world # ENH >>>>> git hf release start 0.1.0 >>>>> ## commit (e.g. update __version__, setup.py, release notes) >>>>> git hf release finish 0.1.0 >>>>> git hf release finish 0.1.0 >>>>> git tag | grep 'v0.1.0' >>>>> >>>>> The GitFlow HubFlow illustrations are very helpful for visualizing >>>>> and understanding any DVCS workflow: >>>>> ``__. >>>>> >>>>> >>>>> ****** with HTML formatting ***** >>>>> >>>>> >>>>> git clone ssh://git at github.com/westurner/dotfiles >>>>> # git checkout master >>>>> git hf init >>>>> ## Update versiontag in .git/config to prefix release tags with 'v' >>>>> git config hubflow.prefix.versiontag=v >>>>> #cat .git/config # ... >>>>> # [hubflow "prefix"] >>>>> # feature = feature/ >>>>> # release = release/ >>>>> # hotfix = hotfix/ >>>>> # support = support/ >>>>> # versiontag = v >>>>> # >>>>> git hf feature start ENH_print_hello_world >>>>> ## commit, commit, commit >>>>> git hf feature finish ENH_print_hello_world # ENH >>>>> git hf release start 0.1.0 >>>>> ## commit (e.g. update __version__, setup.py, release notes) >>>>> git hf release finish 0.1.0 >>>>> git hf release finish 0.1.0 >>>>> git tag | grep 'v0.1.0' >>>>> >>>>> >>>>> On Mon, Jun 29, 2015 at 1:15 PM, Brian Granger >>>>> wrote: >>>>> >>>>>> I am +1 on having separate branches for ongoing 4.x work that include >>>>>> more than just bug fixes - especially for the notebook and widgets. I >>>>>> have a slight preference to have master always be the newer stuff, but >>>>>> don't feel too strongly about that. >>>>>> >>>>>> On Mon, Jun 29, 2015 at 10:54 AM, Jonathan Frederic >>>>>> wrote: >>>>>> >> I wanted to +1 the proposal to start creating branches for new >>>>>> versions >>>>>> >> when a feature freeze occurs. >>>>>> > >>>>>> > >>>>>> > +1 here too, I'm going to do that with ipywidgets for SciPy, create >>>>>> a 5.x >>>>>> > branch. There's a lot of stuff I want to get a jump start on, and >>>>>> SciPy is >>>>>> > a great time to do it. I don't want it to end up like numerous >>>>>> other >>>>>> > experiments, which end up getting thrown out and redone completely >>>>>> just >>>>>> > because of stagmentation and rebase difficulty. >>>>>> > >>>>>> > WRT to the documentation debt, I think it's important to remind >>>>>> everyone >>>>>> > that that is intentional! I've looked at adding JS docs a couple >>>>>> times now, >>>>>> > but when I brought it up we've decided as a group that it was lower >>>>>> priority >>>>>> > because we did not want to commit JavaScript APIs that we know will >>>>>> change. >>>>>> > >>>>>> > I think when we figure out how the front-end packaging and component >>>>>> > refactor will work, we definitely want to commit to **something**. >>>>>> > >>>>>> > On Mon, Jun 29, 2015 at 7:04 AM, Sylvain Corlay < >>>>>> sylvain.corlay at gmail.com> >>>>>> > wrote: >>>>>> >> >>>>>> >> I wanted to +1 the proposal to start creating branches for new >>>>>> versions >>>>>> >> when a feature freeze occurs. Independently of the discussion on >>>>>> phosphor, I >>>>>> >> completely agree with Min on the diagnosis that there is not enough >>>>>> >> available parallel work. >>>>>> >> >>>>>> >> Regarding phosphor and the work on refactoring the front-end, >>>>>> thanks for >>>>>> >> creating the centralized phosphor notebook repository in the >>>>>> organization. I >>>>>> >> did some experiments lately with the widgets and did not know >>>>>> where this >>>>>> >> could fall, or how to share it without requiring it to install >>>>>> phosphor etc. >>>>>> >> Coordination is also important for new developments, even when >>>>>> they have not >>>>>> >> yet achieved the stability of the main components of the project. >>>>>> >> >>>>>> >> Best, >>>>>> >> >>>>>> >> Sylvain >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> On Mon, Jun 29, 2015 at 9:25 AM, Jason Grout >>>>>> wrote: >>>>>> >>> >>>>>> >>> On 6/26/15 19:45, Fernando Perez wrote: >>>>>> >>> >>>>>> >>> While I hear very much the spirit of what you are saying, and I >>>>>> >>> certainly think that we can't lose sight that the *only* thing >>>>>> that >>>>>> >>> ultimately matters is whether we serve our users well or not, >>>>>> there's a >>>>>> >>> big piece that is already burning under us that probably can't >>>>>> wait. In >>>>>> >>> fact, at the last dev meeting, Jason already posted his new draft >>>>>> code >>>>>> >>> in this direction: >>>>>> >>> >>>>>> >>> https://github.com/jasongrout/phosphor-notebook >>>>>> >>> >>>>>> >>> >>>>>> >>> I just wanted to mention that I support what Fernando, Brian, and >>>>>> Chris >>>>>> >>> have said about moving forward with refactoring the notebook. >>>>>> We're making >>>>>> >>> good progress, even while still ramping up. For example, Steven >>>>>> Silvester >>>>>> >>> has put a lot of work recently in porting over the kernel >>>>>> javascript to >>>>>> >>> Typescript and phosphor (along with dependencies): >>>>>> >>> >>>>>> >>> https://github.com/jasongrout/phosphor-notebook/pull/2 >>>>>> >>> >>>>>> >>> I just put in an in-progress pull request for documenting the API >>>>>> for >>>>>> >>> kernels, kernelspecs, and sessions (which I realized when looking >>>>>> at the >>>>>> >>> kernel javascript file was woefully undocumented/incorrectly >>>>>> documented): >>>>>> >>> https://github.com/jupyter/notebook/pull/173. This shows our >>>>>> refactoring >>>>>> >>> work is also having an immediate direct impact on the current >>>>>> notebook as >>>>>> >>> well. >>>>>> >>> >>>>>> >>> In another message on this thread, Min suggested having a 5.x >>>>>> branch for >>>>>> >>> further development, like the phosphor notebook. For now, I >>>>>> think the >>>>>> >>> phosphor notebook can proceed as a separate project - it's >>>>>> totally a >>>>>> >>> front-end thing at this point, and we're doing enough code >>>>>> clean-up and >>>>>> >>> rewriting from js to typescript that I think it's all right to >>>>>> start in a >>>>>> >>> fresh repo. Which brings up another point: can we make an >>>>>> official Jupyter >>>>>> >>> repo for the phosphor notebook work, rather than using my >>>>>> personal repo? >>>>>> >>> I'm happy to continue hosting >>>>>> >>> https://github.com/jasongrout/phosphor-notebook/ in my personal >>>>>> github >>>>>> >>> account for the time being, or set up a temporary organization so >>>>>> we can >>>>>> >>> collaborate more effectively, but I think it would make more >>>>>> sense to bump >>>>>> >>> it up to an experimental repo in the jupyter github organization, >>>>>> developed >>>>>> >>> in parallel with the current notebook. >>>>>> >>> >>>>>> >>> Thomas, one thing to consider is that us working on a phosphor >>>>>> notebook >>>>>> >>> doesn't preclude interested people from enhancing the existing >>>>>> notebook in >>>>>> >>> the short term. We'd like the phosphor notebook to get to a >>>>>> comparable >>>>>> >>> state with the current notebook as quickly as possible, but it >>>>>> will still >>>>>> >>> take some time. >>>>>> >>> >>>>>> >>> Also, I totally agree with Thomas that dogfooding the notebook >>>>>> (and >>>>>> >>> watching/helping others actually use it to get work done) is >>>>>> *extremely* >>>>>> >>> important to understanding what we want here. And I also agree >>>>>> with others >>>>>> >>> on this thread that documentation is sorely lacking. We'll be >>>>>> working on >>>>>> >>> that in the phosphor notebook as we go along too. >>>>>> >>> >>>>>> >>> Thanks, >>>>>> >>> >>>>>> >>> Jason >>>>>> >>> >>>>>> >>> _______________________________________________ >>>>>> >>> IPython-dev mailing list >>>>>> >>> IPython-dev at scipy.org >>>>>> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>>> >>> >>>>>> >> >>>>>> >> >>>>>> >> _______________________________________________ >>>>>> >> IPython-dev mailing list >>>>>> >> IPython-dev at scipy.org >>>>>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>>> >> >>>>>> > >>>>>> > >>>>>> > _______________________________________________ >>>>>> > IPython-dev mailing list >>>>>> > IPython-dev at scipy.org >>>>>> > http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>>> > >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Brian E. Granger >>>>>> Cal Poly State University, San Luis Obispo >>>>>> @ellisonbg on Twitter and GitHub >>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com >>>>>> _______________________________________________ >>>>>> IPython-dev mailing list >>>>>> IPython-dev at scipy.org >>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> IPython-dev mailing list >>>>> IPython-dev at scipy.org >>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Jun 29 15:24:14 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 29 Jun 2015 12:24:14 -0700 Subject: [IPython-dev] Notebooks enforce specific kernel In-Reply-To: References: Message-ID: On 29 June 2015 at 10:54, MinRK wrote: > I realize there's an issue with IPython's attempts to pseudo-activate a > virtualenv if one is found but IPython isn't using it. If the notebook > server is in an env and the kernel isn't, the kernel will try to activate > the server's env (right?). Should we: > > 1. remove the attempts to activate envs > 2. pop VIRTUAL_ENV from the env passed down to kernels > I think we should probably still check the virtualenv when you start terminal IPython, but it doesn't make sense for the notebook. What I'm not sure about is the zmq terminal console and Qt console. It's technically easiest to draw the line between the kernel and the single-process interface, but it might be surprising that the zmq consoles behave differently in that regard from the plain terminal interface. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Mon Jun 29 18:16:39 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 29 Jun 2015 15:16:39 -0700 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: On Mon, Jun 29, 2015 at 1:11 AM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > I agree, though do you want to improve our communication or not ? > I'd like to invite you to consider perhaps a different way of tackling this question: I think it should go without saying that we all want to improve our communications. Posing a question in this fashion is effectively questioning our good faith in the project... The issue, instead, is that while we are trying to improve our communications, we're doing so within very limited constraints. So we're trying to find a good balance of time, resources and effectiveness. Regarding the blog, I'm absolutely sure that we do produce enough interesting work for more than one great post per week. But I'm not sure that right now we have the time and energy to *write* that post. And I do know that we may at least be able to write a less-polished, but still-useful ML post that summarizes the state of things. I know that, in the past, I've made a lot of long-ish posts on our MLs that over the years have become referenced many times, and in a better world would actually be blog posts. But I did them only as ML posts, simply due to lack of time... For example, the one about the early days of the notebook was long and detailed enough that eventually I managed to *rewrite* it as a blog post: http://blog.fperez.org/2012/01/ipython-notebook-historical.html but that almost never happens simply b/c I don't have the time. So, as Min tactfully and kindly said above, I'd like to reiterate a request for a bit of patience: we're all trying to do our best, and we're going into this with our best effort under tight constraints. Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Mon Jun 29 18:28:56 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 29 Jun 2015 15:28:56 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: On Mon, Jun 29, 2015 at 11:44 AM, Brian Granger wrote: > Could you start the documentation stuff as a separate thread? This one > is getting a bit difficult to follow (remembering why email is so > painful :( > Actually, I suggest we don't even try :) I think we should conclude this one for now, we have too much stuff in flight and scipy is coming up... I think we're clearly converging on that the refactoring *will happen*. It's just a matter of the team working together at SciPy, face to face, on the logistics of how to make it through the next few months of that work, so that all the key pieces go through smoothly and as quickly as possible. Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Mon Jun 29 20:56:39 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 29 Jun 2015 17:56:39 -0700 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: Message-ID: On Sat, Jun 27, 2015 at 8:07 PM, MinRK wrote: > I think turning Tuesdays into check-ins/reports is a good idea. Having > just one or two topics for actual in-depth meetings with smaller groups > should definitely be more effective. > OK, here's a wrap-up and my suggested path forward, so we can call this thread done, unless someone objects strenuously to something. As usual, we will fine-tune over time as we learn from experience... - Let's go with a max of 5 minutes per person on reporting of activity. - When reporting, you should try make sure what you say answers the following questions for the others: * What have I been working on? * What do I plan on working on? * What things are preventing me from making progress? This will ensure that everyone leaves knowing what everyone is up to, and that we can effectively help each other, identify where resources are needed, bottlenecks, etc. - We try to take notes as usual collectively on Hackpad, BUT, we designate a person each week who has an explicit responsibility that week to spend a bit of time after the meeting making sure the notes make sense and posting them to the Jupyter list. Hopefully that won't take more than a few minutes, if the collective note-taking was sufficient. This will give us a static record on the list of each week's meeting, which is better than just having a doc on hackpad. I volunteer to be the note-poster for tomorrow. - Let's keep the total meeting time to 1h, hard limit. Tomorrow, we're going to use a trick to enforce that: our nice videoconferencing room at BIDS is booked at 11am every other Tuesday, so we'll get kicked out anyways. We could obviously use another one, but instead we'll use the room with the better audio equipment and be forced to stay on schedule :) Did I miss anything? Cheers, f -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Jun 29 21:25:33 2015 From: benjaminrk at gmail.com (MinRK) Date: Mon, 29 Jun 2015 18:25:33 -0700 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: Message-ID: On Mon, Jun 29, 2015 at 5:56 PM, Fernando Perez wrote: > On Sat, Jun 27, 2015 at 8:07 PM, MinRK wrote: > >> I think turning Tuesdays into check-ins/reports is a good idea. Having >> just one or two topics for actual in-depth meetings with smaller groups >> should definitely be more effective. >> > > OK, here's a wrap-up and my suggested path forward, so we can call this > thread done, unless someone objects strenuously to something. As usual, we > will fine-tune over time as we learn from experience... > > - Let's go with a max of 5 minutes per person on reporting of activity. > > > - When reporting, you should try make sure what you say answers the > following questions for the others: > > * What have I been working on? > * What do I plan on working on? > * What things are preventing me from making progress? > > This will ensure that everyone leaves knowing what everyone is up to, and > that we can effectively help each other, identify where resources are > needed, bottlenecks, etc. > > > - We try to take notes as usual collectively on Hackpad, BUT, we designate > a person each week who has an explicit responsibility that week to spend a > bit of time after the meeting making sure the notes make sense and posting > them to the Jupyter list. Hopefully that won't take more than a few > minutes, if the collective note-taking was sufficient. > > This will give us a static record on the list of each week's meeting, > which is better than just having a doc on hackpad. > > I volunteer to be the note-poster for tomorrow. > > > - Let's keep the total meeting time to 1h, hard limit. Tomorrow, we're > going to use a trick to enforce that: our nice videoconferencing room at > BIDS is booked at 11am every other Tuesday, so we'll get kicked out > anyways. We could obviously use another one, but instead we'll use the > room with the better audio equipment and be forced to stay on schedule :) > > > Did I miss anything? > Sounds like a good plan. See you all tomorrow. -MinRK > > Cheers, > > f > > -- > You received this message because you are subscribed to the Google Groups > "Project Jupyter" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jupyter+unsubscribe at googlegroups.com. > To post to this group, send email to jupyter at googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jupyter/CAHAreOpdZ4tryfYk0PKktwJ2iTbhyMttYoGD9JU3c8A4CdM5mA%40mail.gmail.com > > . > > For more options, visit https://groups.google.com/d/optout. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Tue Jun 30 01:18:22 2015 From: steve at holdenweb.com (Steve Holden) Date: Tue, 30 Jun 2015 06:18:22 +0100 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: <3BD7C224-0C02-4DDF-B48B-56F465A7796A@holdenweb.com> Hi Fernando, On Jun 29, 2015, at 11:16 PM, Fernando Perez wrote: > So, as Min tactfully and kindly said above, I'd like to reiterate a request for a bit of patience: we're all trying to do our best, and we're going into this with our best effort under tight constraints. Those with long memories may remember the monthly "python-dev summaries". While these posts were interesting and useful, they inevitably became a burden for their individual authors as mailing volume grew. From a purely personal point of view I'd rather take what you say on good faith and try to find some way of reporting progress that doesn't distract developers form their primary goal of, rem, developing. regards Steve -- Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / @holdenweb -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Tue Jun 30 02:16:49 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 29 Jun 2015 23:16:49 -0700 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: Message-ID: Sounds good, I have also posted a summary of these changed at the top of the meeting hackpad. See you tomorrow! Cheers, Brian On Mon, Jun 29, 2015 at 6:25 PM, MinRK wrote: > > > On Mon, Jun 29, 2015 at 5:56 PM, Fernando Perez > wrote: >> >> On Sat, Jun 27, 2015 at 8:07 PM, MinRK wrote: >>> >>> I think turning Tuesdays into check-ins/reports is a good idea. Having >>> just one or two topics for actual in-depth meetings with smaller groups >>> should definitely be more effective. >> >> >> OK, here's a wrap-up and my suggested path forward, so we can call this >> thread done, unless someone objects strenuously to something. As usual, we >> will fine-tune over time as we learn from experience... >> >> - Let's go with a max of 5 minutes per person on reporting of activity. >> >> >> - When reporting, you should try make sure what you say answers the >> following questions for the others: >> >> * What have I been working on? >> * What do I plan on working on? >> * What things are preventing me from making progress? >> >> This will ensure that everyone leaves knowing what everyone is up to, and >> that we can effectively help each other, identify where resources are >> needed, bottlenecks, etc. >> >> >> - We try to take notes as usual collectively on Hackpad, BUT, we designate >> a person each week who has an explicit responsibility that week to spend a >> bit of time after the meeting making sure the notes make sense and posting >> them to the Jupyter list. Hopefully that won't take more than a few minutes, >> if the collective note-taking was sufficient. >> >> This will give us a static record on the list of each week's meeting, >> which is better than just having a doc on hackpad. >> >> I volunteer to be the note-poster for tomorrow. >> >> >> - Let's keep the total meeting time to 1h, hard limit. Tomorrow, we're >> going to use a trick to enforce that: our nice videoconferencing room at >> BIDS is booked at 11am every other Tuesday, so we'll get kicked out anyways. >> We could obviously use another one, but instead we'll use the room with the >> better audio equipment and be forced to stay on schedule :) >> >> >> Did I miss anything? > > > Sounds like a good plan. See you all tomorrow. > > -MinRK > >> >> >> Cheers, >> >> f >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Project Jupyter" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to jupyter+unsubscribe at googlegroups.com. >> To post to this group, send email to jupyter at googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/jupyter/CAHAreOpdZ4tryfYk0PKktwJ2iTbhyMttYoGD9JU3c8A4CdM5mA%40mail.gmail.com. >> >> For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "Project Jupyter" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jupyter+unsubscribe at googlegroups.com. > To post to this group, send email to jupyter at googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jupyter/CAHNn8BV61o_M8ziSOQnfOJz0Am6Pux14Yk6TVumCBK28LsaMVw%40mail.gmail.com. > > For more options, visit https://groups.google.com/d/optout. -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From tony.hirst at open.ac.uk Tue Jun 30 07:45:21 2015 From: tony.hirst at open.ac.uk (Tony.Hirst) Date: Tue, 30 Jun 2015 11:45:21 +0000 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... Message-ID: A site I have found useful for keeping tabs on what's going on in the R-universe is http://www.r-bloggers.com/ which aggregates posts from approved feeds (eg I have an Rstats category feed from my more general blog on that I use to syndicate on topic posts into that context). Is there a similar initiative around the IPython ecosystem? (I've not seen one?) I also keep wondering if more could be made of nbviewer? This could provide the basis of a regular round up post ("This week's notable notebooks on nbviewer")? (On a side note, I've often wondered about how I migh tbe able to turn nbviewer into a pseudo blogging platfrom, eg by scraping http://nbviewer.ipython.org/gist/MYACCOUNT and producing a feed from it... In that respect, it would be really handy if I could tag notebooks and see the tags on that page (though a pseudo-tag could be hacked using eg tag:MYTAG in the description that could be easily retrieved.) tony -- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302). The Open University is authorised and regulated by the Financial Conduct Authority. From ssanderson at quantopian.com Tue Jun 30 09:20:22 2015 From: ssanderson at quantopian.com (Scott Sanderson) Date: Tue, 30 Jun 2015 06:20:22 -0700 (PDT) Subject: [IPython-dev] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: +1 for posting short write-ups of Gitter conversations. I wrote up and example of what I think would be helpful both as a user and as a sometimes developer here: https://groups.google.com/forum/#!topic/jupyter/COhsW7rZgLc. The basic format of the post is: 1. Here's the problem I have. 2. Here's my current best solution 3. Here's what's good/bad about the current solution. Having a clear problem and solution statement is useful users who come along later trying to solve the same problem. Having a review of the strengths and weaknesses of the solution helps developers better understand what the current strong/weak points are in the existing APIs. Hope this is helpful, - Scott On Saturday, June 27, 2015 at 7:34:05 PM UTC-4, Fernando Perez wrote: > > Hi all, > > sorry for the cross-post, but this is precisely about project-wide > communication. Probably best to keep the thread on the Jupyter list. > > As the project has grown, we've tried to adapt our communications channels > both to serve the community and to manage our limited resources. Time for > email is scarce (I know I'm not the only one perennially behind), and for > certain problems real-time conversation is extremely valuable. > > So we've added gitter to our mix of tools, and in some cases our gitter > rooms are very useful. But we're also finding a problem: it's proving to > be very hard to follow the "big picture" of the project if you're not > directly participating in those conversations. Even for *me*, since these > days I unfortunately rarely have that kind of time, many parts of the > project have become quite opaque, and reading gitter logs is not the same > as reading an email archive. While it's perfectly viable to read a long > and complex email thread to catch up on a discussion after the fact, doing > so with a real-time chat log is, in practice, nearly impossible. > > So, we'd like to fine-tune our communication practices, so that we hit a > better balance of having a "slow record" over asynchronous email, while > allowing for the more rapid-fire discussion that real-time channels like > gitter allow. > > The tools at our disposal are: > > - Our mailing lists (jupyter & IPython-dev) > > - Our gitter rooms: we don't want to kill them, we just need to adjust > what we want to use them for, and probably reduce the number of rooms. As > the number of repos we have grows, having dozens of rooms is probably not > manageable. > > - Github: issues will continue being used by folks as a mechanism to > effectively submit problem reports that, in practice, are often questions. > > - StackOverflow: not to be underestimated, it's archival, searchable, etc. > > > I don't want to dictate policy here, so I'll propose a rough first draft, > to allow for discussion, ideas and fine-tuning: > > > - We try to do a better job of communicating important ideas, questions, > discussions, etc, on the project lists. Let's encourage our users to do > the same. We also post key announcements more regularly here. Everyone > should really feel comfortable > > - We stop pointing people to gitter as the *first* help stop. We should > point them to the mailing list (and? OR?) StackOverflow instead. Having a > non-archival medium be the first point of help means a vast amount of > duplicated and wasted time helping people. > > - The Gitter real-time help room and repo rooms can remain active but we > try to maintain a discipline of using them only for when folks need the > real-time collaboration. If key decisions are made here, let's try to have > a discipline of periodically posting a quick summary, even if it's just a > very short one, to the mailing list, pointing out what happened. In the > long run, it will help us all better keep in touch with what the project is > doing. > > > What other ideas do people have, that can help us better communicate, both > among project regulars and with the more occasional, broader community > participants? > > Cheers > > f > > > > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Tue Jun 30 10:38:35 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Tue, 30 Jun 2015 07:38:35 -0700 Subject: [IPython-dev] Single click copy+paste in IPython notebook? In-Reply-To: References: Message-ID: <518D5C95-43B2-4022-8283-B31C5C365BD8@gmail.com> > On Jun 27, 2015, at 11:13, Anton Suchaneck wrote: > > I'm not at scipy, but I try to keep up with videos. I'll be excited to watch the video. > I'd also appreciate any pointers to documentation I might need for this particular task and the IPython hooks (classes, methods, ...) I need to accomplish a copy/paste extension. Docs, especially on JS side are bad. Moving to typescript would help. Click and focus things on browser are so annoying that I can?t tell you the best way to go for it. I believe only mostly Linux user will be interested as it is uncommon to have middle click past on anything but linux/unix. -- MD > Or what do you think: Is it more likely or easier to introduce such a copy/paste functionality rather as a browser plugin or even globally in the OS (Windows or Linux)? > > Thanks Matthias! > > 2015-06-26 20:20 GMT+02:00 Matthias Bussonnier >: > Are you at scipy? Otherwise I think our talk on extensions Will be recorded, and we'll put the material online. > > Sent from my iPhone > > On 26 juin 2015, at 18:58, Anton Suchaneck > wrote: > >> So it would be possible? >> I currently only know Python and have no idea about internals of the notebook, Javascript or so. >> Could you give me a pointer what I need to learn and where I need to hook in to make this feature? >> >> 2015-06-26 18:55 GMT+02:00 Thomas Kluyver >: >> You could have a go at doing that in an extension. It's probably too fragile to include in the notebook by default, given all the other things that middle click can mean. >> >> Thomas >> >> On 26 June 2015 at 09:26, Anton Suchaneck > wrote: >> Hello! >> >> I'd like to use single click copy+paste in the IPython notebook - as it is found in some advanced editors. >> Single click means: >> * My cursor is at the destination position >> * I press the middle mouse and start selecting text at some other place. Meanwhile the actual cursor isn't moving >> * as soon as I release the middle mouse now, the selected text is inserted at the position where my curser was at the beginning. >> I used only a single mouse click and no keyboard for this. >> >> Is it possible to include this into the IPython notebook? >> Where would that be (extension, codemirror, browser, OS, ...)? >> >> Cheers, >> Anton >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Tue Jun 30 10:55:15 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Tue, 30 Jun 2015 07:55:15 -0700 Subject: [IPython-dev] Proposal: soft moratorium on re-architecting for 5.0 In-Reply-To: References: Message-ID: <963F3253-CE7E-41F6-BDAA-B0DFB9B87DB1@gmail.com> > On Jun 29, 2015, at 05:44, Nicholas Bollweg wrote: > > For why, see the Mathematica documentation, which is really a better yardstick than other open source projects. > Yes we want to do that, though cross language and cross library is not easy. Mathematica has the advantage that they do not really distinguish the language from the sodlib. > On Jun 29, 2015, at 06:59, Kyle Kelley wrote: > > Gosh, I'd rather read foo(path:string, notebookName:string) any day, instead of having to *also* reference aliases infer based on the name itself. Well you see what I mean, the types helps, of course naming arguments is necessary. But for a name like action_name_is_or_callback is not super nice. The types in typescript docs are also link so it?s easy to read, and you can get deeper details without looking at the source. -- M (gosh, I was only in a plane for 11h, and got 180+ mails opening my laptop, so sorry if I skim through) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Tue Jun 30 11:32:22 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Tue, 30 Jun 2015 08:32:22 -0700 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: Hi Tony, > On Jun 30, 2015, at 04:45, Tony.Hirst wrote: > > A site I have found useful for keeping tabs on what's going on in the > R-universe is http://www.r-bloggers.com/ which aggregates posts from > approved feeds (eg I have an Rstats category feed from my more general > blog on that I use to syndicate on topic posts into that context). > > Is there a similar initiative around the IPython ecosystem? (I've not seen > one?) Not really, I have seen the idea a few time, No one have put it in place. We also probably need someone to take care of it, if there are abuses. > I also keep wondering if more could be made of nbviewer? This could > provide the basis of a regular round up post ("This week's notable > notebooks on nbviewer?)? > > (On a side note, I've often wondered about how I migh tbe able to turn > nbviewer into a pseudo blogging platfrom, eg by scraping > http://nbviewer.ipython.org/gist/MYACCOUNT and producing a feed from it... > In that respect, it would be really handy if I could tag notebooks and see > the tags on that page (though a pseudo-tag could be hacked using eg > tag:MYTAG in the description that could be easily retrieved.) This might require a ?persistence? on nbviewer, but right now nbviewer is stateless, though, if you like to send PR and have ideas... I use (sometimes) nikola, which can take a bunch on ipynb and make a blog from it on ghpages. it (should) auto build on travis-ci when I push my ipynb repo. And this solve one of the problem above about caching, as the rebuilt is triggered on Push and there is no need to scrape. -- M > > tony > > > > > -- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302). The Open University is authorised and regulated by the Financial Conduct Authority. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From bussonniermatthias at gmail.com Tue Jun 30 12:07:27 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Tue, 30 Jun 2015 09:07:27 -0700 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: Message-ID: <077F4F6B-0E87-4326-BB38-F3757C91E546@gmail.com> Last minutes change of plan (Probably) new furniture beeing installed in video-room. So no BlueJeans setup. We will have to fallback on something else most likely. -- M > On Jun 29, 2015, at 23:16, Brian Granger wrote: > > Sounds good, I have also posted a summary of these changed at the top > of the meeting hackpad. See you tomorrow! > > Cheers, > > Brian > > On Mon, Jun 29, 2015 at 6:25 PM, MinRK wrote: >> >> >> On Mon, Jun 29, 2015 at 5:56 PM, Fernando Perez >> wrote: >>> >>> On Sat, Jun 27, 2015 at 8:07 PM, MinRK wrote: >>>> >>>> I think turning Tuesdays into check-ins/reports is a good idea. Having >>>> just one or two topics for actual in-depth meetings with smaller groups >>>> should definitely be more effective. >>> >>> >>> OK, here's a wrap-up and my suggested path forward, so we can call this >>> thread done, unless someone objects strenuously to something. As usual, we >>> will fine-tune over time as we learn from experience... >>> >>> - Let's go with a max of 5 minutes per person on reporting of activity. >>> >>> >>> - When reporting, you should try make sure what you say answers the >>> following questions for the others: >>> >>> * What have I been working on? >>> * What do I plan on working on? >>> * What things are preventing me from making progress? >>> >>> This will ensure that everyone leaves knowing what everyone is up to, and >>> that we can effectively help each other, identify where resources are >>> needed, bottlenecks, etc. >>> >>> >>> - We try to take notes as usual collectively on Hackpad, BUT, we designate >>> a person each week who has an explicit responsibility that week to spend a >>> bit of time after the meeting making sure the notes make sense and posting >>> them to the Jupyter list. Hopefully that won't take more than a few minutes, >>> if the collective note-taking was sufficient. >>> >>> This will give us a static record on the list of each week's meeting, >>> which is better than just having a doc on hackpad. >>> >>> I volunteer to be the note-poster for tomorrow. >>> >>> >>> - Let's keep the total meeting time to 1h, hard limit. Tomorrow, we're >>> going to use a trick to enforce that: our nice videoconferencing room at >>> BIDS is booked at 11am every other Tuesday, so we'll get kicked out anyways. >>> We could obviously use another one, but instead we'll use the room with the >>> better audio equipment and be forced to stay on schedule :) >>> >>> >>> Did I miss anything? >> >> >> Sounds like a good plan. See you all tomorrow. >> >> -MinRK >> >>> >>> >>> Cheers, >>> >>> f >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Project Jupyter" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to jupyter+unsubscribe at googlegroups.com. >>> To post to this group, send email to jupyter at googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/jupyter/CAHAreOpdZ4tryfYk0PKktwJ2iTbhyMttYoGD9JU3c8A4CdM5mA%40mail.gmail.com. >>> >>> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Project Jupyter" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to jupyter+unsubscribe at googlegroups.com. >> To post to this group, send email to jupyter at googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/jupyter/CAHNn8BV61o_M8ziSOQnfOJz0Am6Pux14Yk6TVumCBK28LsaMVw%40mail.gmail.com. >> >> For more options, visit https://groups.google.com/d/optout. > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > > -- > You received this message because you are subscribed to the Google Groups "Project Jupyter" group. > To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscribe at googlegroups.com. > To post to this group, send email to jupyter at googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAH4pYpSLq5LqHgoX1ULsqw3EwB%3Dn%2BR0qiqYKE%2BrHb2R2JEQSLg%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. From jason at jasongrout.org Tue Jun 30 12:21:28 2015 From: jason at jasongrout.org (Jason Grout) Date: Tue, 30 Jun 2015 16:21:28 +0000 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: <077F4F6B-0E87-4326-BB38-F3757C91E546@gmail.com> References: <077F4F6B-0E87-4326-BB38-F3757C91E546@gmail.com> Message-ID: We could set up an audio conference call if needed. Let us know if we need to. Jason On Tue, Jun 30, 2015, 12:07 Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > Last minutes change of plan (Probably) new furniture beeing installed in > video-room. > So no BlueJeans setup. > We will have to fallback on something else most likely. > > -- > M > > > On Jun 29, 2015, at 23:16, Brian Granger wrote: > > > > Sounds good, I have also posted a summary of these changed at the top > > of the meeting hackpad. See you tomorrow! > > > > Cheers, > > > > Brian > > > > On Mon, Jun 29, 2015 at 6:25 PM, MinRK wrote: > >> > >> > >> On Mon, Jun 29, 2015 at 5:56 PM, Fernando Perez > >> wrote: > >>> > >>> On Sat, Jun 27, 2015 at 8:07 PM, MinRK wrote: > >>>> > >>>> I think turning Tuesdays into check-ins/reports is a good idea. Having > >>>> just one or two topics for actual in-depth meetings with smaller > groups > >>>> should definitely be more effective. > >>> > >>> > >>> OK, here's a wrap-up and my suggested path forward, so we can call this > >>> thread done, unless someone objects strenuously to something. As > usual, we > >>> will fine-tune over time as we learn from experience... > >>> > >>> - Let's go with a max of 5 minutes per person on reporting of > activity. > >>> > >>> > >>> - When reporting, you should try make sure what you say answers the > >>> following questions for the others: > >>> > >>> * What have I been working on? > >>> * What do I plan on working on? > >>> * What things are preventing me from making progress? > >>> > >>> This will ensure that everyone leaves knowing what everyone is up to, > and > >>> that we can effectively help each other, identify where resources are > >>> needed, bottlenecks, etc. > >>> > >>> > >>> - We try to take notes as usual collectively on Hackpad, BUT, we > designate > >>> a person each week who has an explicit responsibility that week to > spend a > >>> bit of time after the meeting making sure the notes make sense and > posting > >>> them to the Jupyter list. Hopefully that won't take more than a few > minutes, > >>> if the collective note-taking was sufficient. > >>> > >>> This will give us a static record on the list of each week's meeting, > >>> which is better than just having a doc on hackpad. > >>> > >>> I volunteer to be the note-poster for tomorrow. > >>> > >>> > >>> - Let's keep the total meeting time to 1h, hard limit. Tomorrow, we're > >>> going to use a trick to enforce that: our nice videoconferencing room > at > >>> BIDS is booked at 11am every other Tuesday, so we'll get kicked out > anyways. > >>> We could obviously use another one, but instead we'll use the room > with the > >>> better audio equipment and be forced to stay on schedule :) > >>> > >>> > >>> Did I miss anything? > >> > >> > >> Sounds like a good plan. See you all tomorrow. > >> > >> -MinRK > >> > >>> > >>> > >>> Cheers, > >>> > >>> f > >>> > >>> -- > >>> You received this message because you are subscribed to the Google > Groups > >>> "Project Jupyter" group. > >>> To unsubscribe from this group and stop receiving emails from it, send > an > >>> email to jupyter+unsubscribe at googlegroups.com. > >>> To post to this group, send email to jupyter at googlegroups.com. > >>> To view this discussion on the web visit > >>> > https://groups.google.com/d/msgid/jupyter/CAHAreOpdZ4tryfYk0PKktwJ2iTbhyMttYoGD9JU3c8A4CdM5mA%40mail.gmail.com > . > >>> > >>> For more options, visit https://groups.google.com/d/optout. > >> > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Project Jupyter" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to jupyter+unsubscribe at googlegroups.com. > >> To post to this group, send email to jupyter at googlegroups.com. > >> To view this discussion on the web visit > >> > https://groups.google.com/d/msgid/jupyter/CAHNn8BV61o_M8ziSOQnfOJz0Am6Pux14Yk6TVumCBK28LsaMVw%40mail.gmail.com > . > >> > >> For more options, visit https://groups.google.com/d/optout. > > > > > > > > -- > > Brian E. Granger > > Cal Poly State University, San Luis Obispo > > @ellisonbg on Twitter and GitHub > > bgranger at calpoly.edu and ellisonbg at gmail.com > > > > -- > > You received this message because you are subscribed to the Google > Groups "Project Jupyter" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to jupyter+unsubscribe at googlegroups.com. > > To post to this group, send email to jupyter at googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/jupyter/CAH4pYpSLq5LqHgoX1ULsqw3EwB%3Dn%2BR0qiqYKE%2BrHb2R2JEQSLg%40mail.gmail.com > . > > For more options, visit https://groups.google.com/d/optout. > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Tue Jun 30 13:09:53 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 30 Jun 2015 10:09:53 -0700 Subject: [IPython-dev] [jupyter] Experimenting with a modified dev meeting format? In-Reply-To: References: <077F4F6B-0E87-4326-BB38-F3757C91E546@gmail.com> Message-ID: We are using bluejeans from another room - seems to be working fine On Tue, Jun 30, 2015 at 9:21 AM, Jason Grout wrote: > We could set up an audio conference call if needed. Let us know if we need > to. > > Jason > > > On Tue, Jun 30, 2015, 12:07 Matthias Bussonnier > wrote: >> >> Last minutes change of plan (Probably) new furniture beeing installed in >> video-room. >> So no BlueJeans setup. >> We will have to fallback on something else most likely. >> >> -- >> M >> >> > On Jun 29, 2015, at 23:16, Brian Granger wrote: >> > >> > Sounds good, I have also posted a summary of these changed at the top >> > of the meeting hackpad. See you tomorrow! >> > >> > Cheers, >> > >> > Brian >> > >> > On Mon, Jun 29, 2015 at 6:25 PM, MinRK wrote: >> >> >> >> >> >> On Mon, Jun 29, 2015 at 5:56 PM, Fernando Perez >> >> wrote: >> >>> >> >>> On Sat, Jun 27, 2015 at 8:07 PM, MinRK wrote: >> >>>> >> >>>> I think turning Tuesdays into check-ins/reports is a good idea. >> >>>> Having >> >>>> just one or two topics for actual in-depth meetings with smaller >> >>>> groups >> >>>> should definitely be more effective. >> >>> >> >>> >> >>> OK, here's a wrap-up and my suggested path forward, so we can call >> >>> this >> >>> thread done, unless someone objects strenuously to something. As >> >>> usual, we >> >>> will fine-tune over time as we learn from experience... >> >>> >> >>> - Let's go with a max of 5 minutes per person on reporting of >> >>> activity. >> >>> >> >>> >> >>> - When reporting, you should try make sure what you say answers the >> >>> following questions for the others: >> >>> >> >>> * What have I been working on? >> >>> * What do I plan on working on? >> >>> * What things are preventing me from making progress? >> >>> >> >>> This will ensure that everyone leaves knowing what everyone is up to, >> >>> and >> >>> that we can effectively help each other, identify where resources are >> >>> needed, bottlenecks, etc. >> >>> >> >>> >> >>> - We try to take notes as usual collectively on Hackpad, BUT, we >> >>> designate >> >>> a person each week who has an explicit responsibility that week to >> >>> spend a >> >>> bit of time after the meeting making sure the notes make sense and >> >>> posting >> >>> them to the Jupyter list. Hopefully that won't take more than a few >> >>> minutes, >> >>> if the collective note-taking was sufficient. >> >>> >> >>> This will give us a static record on the list of each week's meeting, >> >>> which is better than just having a doc on hackpad. >> >>> >> >>> I volunteer to be the note-poster for tomorrow. >> >>> >> >>> >> >>> - Let's keep the total meeting time to 1h, hard limit. Tomorrow, >> >>> we're >> >>> going to use a trick to enforce that: our nice videoconferencing room >> >>> at >> >>> BIDS is booked at 11am every other Tuesday, so we'll get kicked out >> >>> anyways. >> >>> We could obviously use another one, but instead we'll use the room >> >>> with the >> >>> better audio equipment and be forced to stay on schedule :) >> >>> >> >>> >> >>> Did I miss anything? >> >> >> >> >> >> Sounds like a good plan. See you all tomorrow. >> >> >> >> -MinRK >> >> >> >>> >> >>> >> >>> Cheers, >> >>> >> >>> f >> >>> >> >>> -- >> >>> You received this message because you are subscribed to the Google >> >>> Groups >> >>> "Project Jupyter" group. >> >>> To unsubscribe from this group and stop receiving emails from it, send >> >>> an >> >>> email to jupyter+unsubscribe at googlegroups.com. >> >>> To post to this group, send email to jupyter at googlegroups.com. >> >>> To view this discussion on the web visit >> >>> >> >>> https://groups.google.com/d/msgid/jupyter/CAHAreOpdZ4tryfYk0PKktwJ2iTbhyMttYoGD9JU3c8A4CdM5mA%40mail.gmail.com. >> >>> >> >>> For more options, visit https://groups.google.com/d/optout. >> >> >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "Project Jupyter" group. >> >> To unsubscribe from this group and stop receiving emails from it, send >> >> an >> >> email to jupyter+unsubscribe at googlegroups.com. >> >> To post to this group, send email to jupyter at googlegroups.com. >> >> To view this discussion on the web visit >> >> >> >> https://groups.google.com/d/msgid/jupyter/CAHNn8BV61o_M8ziSOQnfOJz0Am6Pux14Yk6TVumCBK28LsaMVw%40mail.gmail.com. >> >> >> >> For more options, visit https://groups.google.com/d/optout. >> > >> > >> > >> > -- >> > Brian E. Granger >> > Cal Poly State University, San Luis Obispo >> > @ellisonbg on Twitter and GitHub >> > bgranger at calpoly.edu and ellisonbg at gmail.com >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Project Jupyter" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an email to jupyter+unsubscribe at googlegroups.com. >> > To post to this group, send email to jupyter at googlegroups.com. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msgid/jupyter/CAH4pYpSLq5LqHgoX1ULsqw3EwB%3Dn%2BR0qiqYKE%2BrHb2R2JEQSLg%40mail.gmail.com. >> > For more options, visit https://groups.google.com/d/optout. >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From takowl at gmail.com Tue Jun 30 14:22:01 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 30 Jun 2015 11:22:01 -0700 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: On 30 June 2015 at 04:45, Tony.Hirst wrote: > A site I have found useful for keeping tabs on what's going on in the > R-universe is http://www.r-bloggers.com/ which aggregates posts from > approved feeds (eg I have an Rstats category feed from my more general > blog on that I use to syndicate on topic posts into that context). > > Is there a similar initiative around the IPython ecosystem? (I've not seen > one?) > These collections of blogs are often called planets. There's a Planet SciPy (http://planet.scipy.org/ ), though the style is not especially modern. We don't have one specific to Jupyter/IPython, but maybe it's something we should consider. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Jun 30 14:40:21 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 30 Jun 2015 11:40:21 -0700 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: <3BD7C224-0C02-4DDF-B48B-56F465A7796A@holdenweb.com> References: <3BD7C224-0C02-4DDF-B48B-56F465A7796A@holdenweb.com> Message-ID: On Mon, Jun 29, 2015 at 10:18 PM, Steve Holden wrote: > Those with long memories may remember the monthly "python-dev summaries". > > While these posts were interesting and useful, they inevitably became a > burden for their individual authors as mailing volume grew. > > From a purely personal point of view I'd rather take what you say on good > faith and try to find some way of reporting progress that doesn't distract > developers form their primary goal of, rem, developing. > Thanks for the cautionary tale from hard-won experience, Steve. Indeed, we dont' want to turn this into a burden for anyone... I'm thinking simply of something where, if a discussion went on for hours and reached an important conclusion, the acting parties take only a few minutes to make a quick recap and post a light summary of it, no more, to the list to keep the rest of us in the loop. I think it's not much to ask, and it's important so that we don't lose that common weave of community knowledge of what's happening across the project. But it's indeed critical that we don't set a bar so high that it becomes burdensome. I know far too well how on these kinds of things, the perfect is the enemy of the good, so we're going to keep the bar low: basic info, links that provide context to be useful and that help us weave the fabric of knowledge, and that keep the burden as minimal as absolutely possible. Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Tue Jun 30 15:45:26 2015 From: steve at holdenweb.com (Steve Holden) Date: Tue, 30 Jun 2015 20:45:26 +0100 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: <3BD7C224-0C02-4DDF-B48B-56F465A7796A@holdenweb.com> Message-ID: <053516DF-F6BD-4E01-BF38-CE89CAB42125@holdenweb.com> On Jun 30, 2015, at 7:40 PM, Fernando Perez wrote: > I'm thinking simply of something where, if a discussion went on for hours and reached an important conclusion, the acting parties take only a few minutes to make a quick recap and post a light summary of it, no more, to the list to keep the rest of us in the loop. > That sounds like a really sensible use of time. Best immediately after the meeting, as otherwise the un-performed task can be a burden. It would probably only mean people scheduling 1:15 for the meeting instead of 1:00, just to make sure commitments and important decisions were recorded. > I think it's not much to ask, and it's important so that we don't lose that common weave of community knowledge of what's happening across the project. But it's indeed critical that we don't set a bar so high that it becomes burdensome. > I think we are in violent agreement :) > I know far too well how on these kinds of things, the perfect is the enemy of the good, so we're going to keep the bar low: basic info, links that provide context to be useful and that help us weave the fabric of knowledge, and that keep the burden as minimal as absolutely possible. > Right. Quit apart from anything else, a) Many hands make light work, and b) Nobody wants to join a project with a reputation for becoming a millstone around people's necks :-) > Cheers, > > f > Always great to hear from you. S -- Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / @holdenweb -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Tue Jun 30 16:16:27 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Tue, 30 Jun 2015 13:16:27 -0700 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: <053516DF-F6BD-4E01-BF38-CE89CAB42125@holdenweb.com> References: <3BD7C224-0C02-4DDF-B48B-56F465A7796A@holdenweb.com> <053516DF-F6BD-4E01-BF38-CE89CAB42125@holdenweb.com> Message-ID: <72435BF5-7EE0-4F05-93F5-522F38C68C07@gmail.com> Hi all, This morning meeting was great, much more efficient than usual. the recap is on ipython.hackpad.com . We?ll curate it a bit and post it on the ML. We?ll also post the video at some point. You can of course jump on the hackpad now and help curate a bit, fix typos and links, ask precisions of what?s unclear. Thanks. -- M > On Jun 30, 2015, at 12:45, Steve Holden wrote: > > On Jun 30, 2015, at 7:40 PM, Fernando Perez > wrote: > >> I'm thinking simply of something where, if a discussion went on for hours and reached an important conclusion, the acting parties take only a few minutes to make a quick recap and post a light summary of it, no more, to the list to keep the rest of us in the loop. >> > That sounds like a really sensible use of time. Best immediately after the meeting, as otherwise the un-performed task can be a burden. It would probably only mean people scheduling 1:15 for the meeting instead of 1:00, just to make sure commitments and important decisions were recorded. > >> I think it's not much to ask, and it's important so that we don't lose that common weave of community knowledge of what's happening across the project. But it's indeed critical that we don't set a bar so high that it becomes burdensome. >> > I think we are in violent agreement :) > >> I know far too well how on these kinds of things, the perfect is the enemy of the good, so we're going to keep the bar low: basic info, links that provide context to be useful and that help us weave the fabric of knowledge, and that keep the burden as minimal as absolutely possible. >> > Right. Quit apart from anything else, > > a) Many hands make light work, and > > b) Nobody wants to join a project with a reputation for becoming a millstone around people's necks :-) > >> Cheers, >> >> f >> > Always great to hear from you. > > S > -- > Steve Holden steve at holdenweb.com / +1 571 484 6266 / +44 208 289 6308 / @holdenweb > > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Jun 30 16:34:37 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 30 Jun 2015 13:34:37 -0700 Subject: [IPython-dev] June 30 Jupyter/IPython Developer meeting minutes Message-ID: Hi all, in the spirit of doing it quickly and decently but without over-burdening us, her we go with the meeting minutes. I'm going to post them both as a copy-paste job straight from HackPad into Gmail (should do HTML) and then below as an export into Markdown. I'd appreciate feedback on whether we should do only HTML, only Markdown or both (or both in separate back to back emails)... I know many folks have moved over to HTML clients, but others still prefer text-only clients, so feedback welcome. Cheers, f ----- HTML format - June 30 Trying with BlueJeans. We recorded the video, it's now posted here: ` https://youtu.be/6yOQnq19Dus`. new faces ! Craig Citro: Employee at Google, joined the CoLaboratory team, was involved in SageMath since its early days, was a postdoc with Sage, worked on Cython. Has known the IPython/Sage devs since ~2005. Min Been teaching in Southampton last week. Worked on getting jupyter_core ready to ship, we're almost there. Getting the dependencies of the QtConsole out of the way. Nobody's blocking me, back from travel. Matthias Bussonnier Was in France for two weeks, worked on SciPy tutorial, need to catch up with Kester and Jason Grout on RealTime work. Kester suggested setting up weekly/biweekly meeting for RT sync. Thomas Kluyver Focusing on teaching material, getting things ready for 4.0. Some changes in jupyter_core, want to release that tomorrow, and then move on next packages that need to be released . Version numbering of different pieces when 4.0 is released - We think slower moving pieces will be +0.1 until branching is necessary, faster moving ones will be +1 straight after release. 4.0 release status - jupyter_core merged some simple changes on Monday. Hope to release tomorrow (Wednesday) unless anything else comes up. Fernando: - Working on teaching materials, didn't get to go to Southampton :-( - Decisions about what communications media we use for what - see email thread. - Blocked, but not by any of us Study on how people use PR: - http://www.gousios.gr/blog/How-contributors-use-pull-requests-GitHub/ - full study http://www.gousios.gr/pub/pullreq-contrib-tr.pdf - 2014 : http://gousios.gr/blog/Exploration-pull-requests/ It's interesting too see what are the pain point globally, cover communication and technical challenge. Interesting to see that most struggle in project is developer responsiveness. Brian Granger Mailing list traffic, hiring: Will Whitney will be working for the summer on completion machinery at Cal Poly. Students also at CalPoly. This coming week there will be big refactoring push at SciPy and lots of discussions, a BoF, etc. Planning for that, ordering stickers, t-shirts, etc. Other than being sick, no big blockers. Nick Bollweg High uptick on issues on Nbviewer, lots of security issues, XSS. Planning on automated exploit testing framework that we can also reuse for tmpnb, our other REST APIs, etc. Been also working with Cyrille Rossant on ipymd support as an alternate content manager to better support metadata. Using yaml custom tags to indicate what's a slide, what's a notebook, etc. Still evolving. Also working on nosebook to integrate notebooks as tests. At workplace, integrating notebooks as semantic storytelling systems https://github.com/bollwyvl/nosebook Jonathan Frederic Working on widget docs and tutorial content for scipy. Also working with Cameron on design scheme for diagrams to unify style across our technical diagrams in general. draw.io is an on-line diagram drawing tool (backed by GDocs, Dropbox, etc.) https://www.draw.io/ Kyle Kelley Went to CodeConf (github's code conference). Lots about Atom & Electron. Talked about web components and met the Atom devs about being able to include the output area for Hydrogen inline within the editor itself, the Atom devs were amenable to that. They also would consider bundling zmq with Atom or Electron, to solve some of the node-gyp installation/update deployment headaches we have right now. Hacking on a sidecar for rich display output, somewhat like the QTConsole. Will Whitney Working on Hydrogen, which is a plugin for Atom that provides integration with the Jupyter kernel protocol. Going to be adding rich info (e.g. type hints) to the autocomplete message spec. jupyter/jupyter_client#38 <- Proposal for message-spec change. Chris Colbert Working on phosphor list and grid controls, scrolling, trees. Mostly working on virtual scrolling (virtual means that it only renders what is visible in the viewport). Performance seems to scale very well (to the quadrillions of rows). https://github.com/phosphorjs/phosphor/blob/feature-listview/src/widgets/listview.ts We need to think of what a 20 second description of Phosphor should be. "In browser tiling windows manager" is unfair and restrictive. At the same time, it's not a full-blown widget toolkit with buttons and sliders etc. It focuses on low-level "scaffolding" layouts and widgets which are hard to get right. Steve Silvester Working for Continuum Analytics with Chris. Adding phosphor signals, cleaning up the callback machinery in the notebook with a "kernel future" object. Will be at Scipy for the meeting. Also working on the Term.js component: took the existing terminado tool and turned into a phosphor terminal, and also took (as a proof of concept) a sidecar and put it into phosphor too. *https://github.com/jupyter/phosphor-notebook* https://github.com/phosphorjs/phosphor-terminal https://github.com/phosphorjs/playground/tree/master/examples/term-sidecar-app Auberon L?pez Work on Spark integration of PySpark, persist some of the configuration in profiles, and creating magic to work with SparkSql. https://github.com/minrk/findspark minrk/findspark#3 https://github.com/alope107/spark-sql-magic Cameron Oelsen Working on updating both IPython and Jupyter websites to have more unified visual languages, support mobile, and unified styling for technical diagrams. Also worked on design of stickers and t-shirts. Need content for what we want the Jupyter site to have on each tab in comparison to what is on the IPython site. Please give input on the hackpad link below: https://jupyter.hackpad.com/Jupyter-Content-and-Organization-Gg50rKW9YW7 Simon Vurens https://github.com/jupyter/spreadsheet Jupyter widget prototype for a spreadsheet using phosphor. A widget to manipulate data, copy/pasting is supported. Using JQuery for now. Ryan Morshead Implementing configurables into matplotlib, looking at where configurable objects could be referenced, looking at all the getter/setter functions in MPL. Jason Grout Working on Phosphor nb and widgets with Chris, Steven, and Sylvain, getting ready for Scipy meeting. ----- Markdown format ## June 30 Trying with BlueJeans. We recorded the video, it's now posted here: `[]( https://youtu.be/6yOQnq19Dus)[https://youtu.be/6yOQnq19Dus](https://youtu.be/6yOQnq19Dus)` . new faces ! Craig Citro: Employee at Google, joined the CoLaboratory team, was involved in SageMath since its early days, was a postdoc with Sage, worked on Cython. Has known the IPython/Sage devs since ~2005. [Min](https://github.com/minrk) Been teaching in Southampton last week. Worked on getting jupyter_core ready to ship, we're almost there. Getting the dependencies of the QtConsole out of the way. Nobody's blocking me, back from travel. [Matthias Bussonnier](https://github.com/carreau/) Was in France for two weeks, worked on SciPy tutorial, need to catch up with Kester and [Jason Grout](/ep/profile/u3HlUfbBz75) on RealTime work. Kester suggested setting up weekly/biweekly meeting for RT sync. [Thomas Kluyver](/ep/profile/oytFGQZl714) Focusing on teaching material, getting things ready for 4.0. Some changes in jupyter_core, want to release that tomorrow, and then move on next packages that need to be released . Version numbering of different pieces when 4.0 is released * We think slower moving pieces will be +0.1 until branching is necessary, faster moving ones will be +1 straight after release. 4.0 release status * jupyter_core merged some simple changes on Monday. Hope to release tomorrow (Wednesday) unless anything else comes up. Fernando: * Working on teaching materials, didn't get to go to Southampton :-( * Decisions about what communications media we use for what - see email thread. * Blocked, but not by any of us Study on how people use PR: - []( http://www.gousios.gr/blog/How-contributors-use-pull-requests-GitHub/)http://www.gousios.gr/blog/How-contributors-use-pull-requests-GitHub/ - full study []( http://www.gousios.gr/pub/pullreq-contrib-tr.pdf)http://www.gousios.gr/pub/pullreq-contrib-tr.pdf - 2014 : []( http://gousios.gr/blog/Exploration-pull-requests/)http://gousios.gr/blog/Exploration-pull-requests/ It's interesting too see what are the pain point globally, cover communication and technical challenge. Interesting to see that most struggle in project is developer responsiveness. [Brian Granger](/ep/profile/BQ7pw5ScLjz) Mailing list traffic, hiring: [Will Whitney](/ep/profile/riDs1o645Fd) will be working for the summer on completion machinery at Cal Poly. Students also at CalPoly. This coming week there will be big refactoring push at SciPy and lots of discussions, a BoF, etc. Planning for that, ordering stickers, t-shirts, etc. Other than being sick, no big blockers. [Nick Bollweg](https://github.com/bollwyvl) High uptick on issues on Nbviewer, lots of security issues, XSS. Planning on automated exploit testing framework that we can also reuse for tmpnb, our other REST APIs, etc. Been also working with Cyrille Rossant on ipymd support as an alternate content manager to better support metadata. Using yaml custom tags to indicate what's a slide, what's a notebook, etc. Still evolving. Also working on nosebook to integrate notebooks as tests. At workplace, integrating notebooks as semantic storytelling systems [](https://github.com/bollwyvl/nosebook)https://github.com/bollwyvl/nosebook [Jonathan Frederic](/ep/profile/FaLk90y9B5J) Working on widget docs and tutorial content for scipy. Also working with Cameron on design scheme for diagrams to unify style across our technical diagrams in general. draw.io is an on-line diagram drawing tool (backed by GDocs, Dropbox, etc.) [](https://www.draw.io/)https://www.draw.io/ [Kyle Kelley](/ep/profile/Gon8Ql8QYJN) Went to CodeConf (github's code conference). Lots about Atom & Electron. Talked about web components and met the Atom devs about being able to include the output area for [Hydrogen]( https://github.com/willwhitney/hydrogen) inline within the editor itself, the Atom devs were amenable to that. They also would consider bundling zmq with Atom or Electron, to solve some of the node-gyp installation/update deployment headaches we have right now. Hacking on a [sidecar]( https://github.com/rgbkrk/jupyter-sidecar) for rich display output, somewhat like the QTConsole. [Will Whitney](/ep/profile/riDs1o645Fd) Working on Hydrogen, which is a plugin for Atom that provides integration with the Jupyter kernel protocol. Going to be adding rich info (e.g. type hints) to the autocomplete message spec. [jupyter/jupyter_client#38]( https://github.com/jupyter/jupyter_client/issues/38) <- Proposal for message-spec change. [Chris Colbert](https://github.com/sccolbert) Working on phosphor list and grid controls, scrolling, trees. Mostly working on virtual scrolling (virtual means that it only renders what is visible in the viewport). Performance seems to scale very well (to the quadrillions of rows). []( https://github.com/phosphorjs/phosphor/blob/feature-listview/src/widgets/listview.ts)https://github.com/phosphorjs/phosphor/blob/feature-listview/src/widgets/listview.ts We need to think of what a 20 second description of Phosphor should be. "In browser tiling windows manager" is unfair and restrictive. At the same time, it's not a full-blown widget toolkit with buttons and sliders etc. It focuses on low-level "scaffolding" layouts and widgets which are hard to get right. [Steve Silvester](https://github.com/blink1073) Working for Continuum Analytics with Chris. Adding phosphor signals, cleaning up the callback machinery in the notebook with a "kernel future" object. Will be at Scipy for the meeting. Also working on the Term.js component: took the existing terminado tool and turned into a phosphor terminal, and also took (as a proof of concept) a [sidecar]( https://github.com/rgbkrk/jupyter-sidecar) and put it into phosphor too. [](https://github.com/jupyter/phosphor-notebook) https://github.com/jupyter/phosphor-notebook []( https://github.com/phosphorjs/phosphor-terminal)https://github.com/phosphorjs/phosphor-terminal []( https://github.com/phosphorjs/playground/tree/master/examples/term-sidecar-app)https://github.com/phosphorjs/playground/tree/master/examples/term-sidecar-app [Auberon L?pez](https://github.com/alope107) Work on Spark integration of PySpark, persist some of the configuration in profiles, and creating magic to work with SparkSql. [](https://github.com/minrk/findspark)https://github.com/minrk/findspark [minrk/findspark#3](https://github.com/minrk/findspark/pull/3) []( https://github.com/alope107/spark-sql-magic)https://github.com/alope107/spark-sql-magic [Cameron Oelsen](https://github.com/cameronoelsen) Working on updating both IPython and Jupyter websites to have more unified visual languages, support mobile, and unified styling for technical diagrams. Also worked on design of stickers and t-shirts. Need content for what we want the Jupyter site to have on each tab in comparison to what is on the IPython site. Please give input on the hackpad link below: []( https://jupyter.hackpad.com/Jupyter-Content-and-Organization-Gg50rKW9YW7)https://jupyter.hackpad.com/Jupyter-Content-and-Organization-Gg50rKW9YW7 [Simon Vurens](https://github.com/svurens) []( https://github.com/jupyter/spreadsheet)https://github.com/jupyter/spreadsheet Jupyter widget prototype for a spreadsheet using phosphor. A widget to manipulate data, copy/pasting is supported. Using JQuery for now. [Ryan Morshead](https://github.com/rmorshea) Implementing configurables into matplotlib, looking at where configurable objects could be referenced, looking at all the getter/setter functions in MPL. [Jason Grout](/ep/profile/u3HlUfbBz75) Working on Phosphor nb and widgets with Chris, Steven, and Sylvain, getting ready for Scipy meeting. -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Tue Jun 30 16:35:14 2015 From: wes.turner at gmail.com (Wes Turner) Date: Tue, 30 Jun 2015 15:35:14 -0500 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: On Tue, Jun 30, 2015 at 10:32 AM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > Hi Tony, > > > > On Jun 30, 2015, at 04:45, Tony.Hirst wrote: > > > > A site I have found useful for keeping tabs on what's going on in the > > R-universe is http://www.r-bloggers.com/ which aggregates posts from > > approved feeds (eg I have an Rstats category feed from my more general > > blog on that I use to syndicate on topic posts into that context). > > > > Is there a similar initiative around the IPython ecosystem? (I've not > seen > > one?) > > Not really, I have seen the idea a few time, No one have put it in place. > We also probably need someone to take care of it, if there are abuses. > > > > I also keep wondering if more could be made of nbviewer? This could > > provide the basis of a regular round up post ("This week's notable > > notebooks on nbviewer?)? > > > > > (On a side note, I've often wondered about how I migh tbe able to turn > > nbviewer into a pseudo blogging platfrom, eg by scraping > > http://nbviewer.ipython.org/gist/MYACCOUNT and producing a feed from > it... > > In that respect, it would be really handy if I could tag notebooks and > see > > the tags on that page (though a pseudo-tag could be hacked using eg > > tag:MYTAG in the description that could be easily retrieved.) > - [ ] RDFa in nbconvert HTML (e.g. schema.org/ Notebook) would be search-indexable - [ ] Currently, one COULD store JSON-LD in notebook metadata * https://wrdrd.com/docs/consulting/knowledge-engineering#rdfa * https://wrdrd.com/docs/consulting/knowledge-engineering#schema-org * http://schema.org/docs/full.html Two URIs for a notebook: - GitHub notebook rendering (./blob/master/path/to/file.py) - nbviewer rendering (e.g. http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures) > > This might require a ?persistence? on nbviewer, but right now nbviewer is > stateless, > though, if you like to send PR and have ideas... > - [ ] Maintain "a gallery of interesting IPython notebooks" as a notebook renderable through nbviewer? https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks > > I use (sometimes) nikola, which can take a bunch on ipynb and make a blog > from it on ghpages. > it (should) auto build on travis-ci when I push my ipynb repo. > > And this solve one of the problem above about caching, as the rebuilt is > triggered on Push and there is no need to scrape. > -- > M > > > > > > tony > > > > > > > > > > -- The Open University is incorporated by Royal Charter (RC 000391), an > exempt charity in England & Wales and a charity registered in Scotland (SC > 038302). The Open University is authorised and regulated by the Financial > Conduct Authority. > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Tue Jun 30 16:37:07 2015 From: wes.turner at gmail.com (Wes Turner) Date: Tue, 30 Jun 2015 15:37:07 -0500 Subject: [IPython-dev] [jupyter] Communications, mailing lists, gitter, etc... In-Reply-To: References: Message-ID: On Tue, Jun 30, 2015 at 1:22 PM, Thomas Kluyver wrote: > On 30 June 2015 at 04:45, Tony.Hirst wrote: > >> A site I have found useful for keeping tabs on what's going on in the >> R-universe is http://www.r-bloggers.com/ which aggregates posts from >> approved feeds (eg I have an Rstats category feed from my more general >> blog on that I use to syndicate on topic posts into that context). >> >> Is there a similar initiative around the IPython ecosystem? (I've not seen >> one?) >> > > These collections of blogs are often called planets. There's a Planet > SciPy (http://planet.scipy.org/ ), though the style is not especially > modern. We don't have one specific to Jupyter/IPython, but maybe it's > something we should consider. > Is there an OPML file? https://github.com/mjibson/goread is not a bad AppEngine Go Atom RSS Reader. > > > Thomas > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: