[Pandas-dev] Mailing list for Python data analytics ecosystem developers?

Thu Jan 3 12:48:22 EST 2019

I got a permission error because it's a private domain (I don't have a
@pydata.org e-mail address)

On Thu, Jan 3, 2019 at 11:46 AM Andy Ray Terrel <andy.terrel at gmail.com> wrote:
>
> It is managed by Google groups.
>
> I can definitely make others admins. Just join up and let me know if you want to admin.
>
> On Thu, Jan 3, 2019 at 11:34 AM Wes McKinney <wesmckinn at gmail.com> wrote:
>>
>> Is it managed by Google groups or is it mailman or Pony Mail? Can you
>> make some of us admins?
>>
>> Thanks
>>
>> On Thu, Jan 3, 2019 at 10:53 AM Andy Ray Terrel <andy.terrel at gmail.com> wrote:
>> >
>> >
>> >
>> > On Mon, Dec 31, 2018 at 12:07 PM Andy Ray Terrel <andy.terrel at gmail.com> wrote:
>> >>
>> >>
>> >> On Mon, Dec 31, 2018 at 11:28 AM Wes McKinney <wesmckinn at gmail.com> wrote:
>> >>>
>> >>> As a discussion list intended for project developers, I am not
>> >>> anticipating so much noise that people become disengaged. If we were
>> >>> creating a forum to collect user feedback, that would be a little bit
>> >>> different. I'm more looking to encourage the sharing of more high
>> >>> level project planning, roadmaps and goals, fund raising activities,
>> >>> and other matters related to the health and growth of the major
>> >>> community projects. It would be really useful for each project to
>> >>> state a list of goals for some future horizon (e.g. 1 year).
>> >>>
>> >>> I have observed that some of these cross-project discussions often
>> >>> only happen in person, or on an ad hoc basis on GitHub issues.
>> >>>
>> >>> User feedback can be helpful, but in practice most projects function
>> >>> as "do-ocracies" where opinions are roughly valued proportional to
>> >>> project contributions.
>> >>>
>> >>> It would also be useful to be able to point users to historical
>> >>> discussions amongst project developers when there are questions or
>> >>> concerns. My anecdotal experience is that the lack of visible /
>> >>> centralized cross-project discussions and roadmapping / planning /
>> >>> goal discussion has at times led to user (or developer) confusion
>> >>> about what different groups of developers are trying to accomplish.
>> >>>
>> >>> The concerns raised seem to be mostly about optimizing large-scale
>> >>> communications. Let's first see if there is communication that needs
>> >>> to be optimized. Even if we add additional tools to facilitate
>> >>> communications, I think we still need a mailing list.
>> >>>
>> >>
>> >> Have we decided which mailing list we desire? I forgot we could also just make it dev at pydata.org if we like. In general I think, we should write up a governance document on pydata as a whole.
>> >>
>> >
>> >
>> > I've created a list dev at pydata.org if we want to use it.
>> >
>> > On the governance front, Leah is putting together a plan around managing pydata conferences going. We also wanted to revamp pydata.org to reflect more of the development community around the ecosystem so definitely send ideas and thoughts.
>> >
>> > -- Andy
>> >
>> >>
>> >> - Andy
>> >>
>> >>>
>> >>> - Wes
>> >>>
>> >>> On Sat, Dec 29, 2018 at 4:58 PM Matthew Rocklin <mrocklin at gmail.com> wrote:
>> >>> >
>> >>> > > I don't see what is wrong with using e-mail.
>> >>> >
>> >>> > There were some issues raised before:
>> >>> >
>> >>> > I'm slightly concerned that a broad ranging e-mail list that encompasses all of PyData would get noisy.  For example I can imagine lengthy conversations on visualization or probabalistic programming that, while I find important, would likely want to take a pass on.  Having a service that includes tags and subscription to those tags may have value.
>> >>> > E-mail list archives tend to collect dust.  If we're doing long-range cross-project coordination then those conversations might have long term value.  We might want to cross reference them, upvote them, subscribe to them, and so on.
>> >>> >
>> >>> > And also some benefits of discourse raised by Nathaniel which might be turned around to be interpreted as concerns with e-mail.
>> >>> >
>> >>> > My impression so far is that discourse takes a bit of adjustment
>> >>> > compared to mailing lists, but it has a lot of valuable features like
>> >>> > multi-quoting, markdown (code blocks, links, ...), solid moderation
>> >>> > tools (e.g. if a discussion diverges you can retroactively split parts
>> >>> > of it out into a new topic), polls (these were incredibly useful for
>> >>> > taking the temperature of the community during the governance
>> >>> > discussions), ability to reply to messages that were posted before you
>> >>> > joined the list, configurable notifications (email me everything /
>> >>> > email me when a new topic is created / email me a summary weekly /
>> >>> > ...), ...
>> >>> >
>> >>> > > It is public, archival, and append-only. GitHub issues are non-archival and comments can be edited or deleted.
>> >>> >
>> >>> > That's certainly true of GitHub issues.  I suspect that it's also true of Discourse (though I'd have to go through the docs to make sure that it wasn't possible to turn it off).  From my perspective the (in)ability to edit or delete comments isn't a big deal.  I'm not particularly concerned with people modifying history in a nefarious way.  Though perhaps my viewpoint here is naive.  I haven't yet run into this issue in our community.
>> >>> >
>> >>> > I think that the biggest benefit to using an e-mail list is that it's a well known technology with a low barrier to adoption.
>> >>> >
>> >>> > I anticipate two likely failure modes for e-mail and discourse respectively:
>> >>> >
>> >>> > EMail: conversation is too diffuse so that people sign up, get bored listening to things that don't interest them, and then stop notifications.  The pydata mailing list ends up being used by small subsets of the community, but not the community as a whole.
>> >>> > Discourse: it's too new/unknown so that no one signs up and it doesn't reach critical mass.  (this seems to be happening with Jupyter's discourse today?)
>> >>> >
>> >>> > There are lots of other pros and cons to each, obviously, but those two outcomes are, I think, the most troublesome.
>> >>> >
>> >>> > On Thu, Dec 27, 2018 at 8:13 AM Wes McKinney <wesmckinn at gmail.com> wrote:
>> >>> >>
>> >>> >> Having dev.pydata.org sounds fine to me.
>> >>> >>
>> >>> >> I don't see what is wrong with using e-mail. It is public, archival,
>> >>> >> and append-only. GitHub issues are non-archival and comments can be
>> >>> >> edited or deleted.
>> >>> >>
>> >>> >> On Thu, Dec 27, 2018 at 6:26 AM Andy Ray Terrel <andy.terrel at gmail.com> wrote:
>> >>> >> >
>> >>> >> > I would recommend we set up a site dev.pydata.org that tells the folks where conversations are happening. While mailing lists are great we might consider just having a github issue tracker set up for cross ecosystem bugs or initiatives. I was planning on decommisionning the numfocus discourse and zulip server as they didn't really have great use. Chris Holdgraf suggested using Topic Box, but user based pricing isn't a great option for open source development.
>> >>> >> >
>> >>> >> > Anywho, both dask and pandas are part of the NumFOCUS projects ecosystem so I'm happy to set anything up for ya'll.
>> >>> >> >
>> >>> >> > -- Andy
>> >>> >> >
>> >>> >> > On Wed, Dec 26, 2018 at 10:35 PM Wes McKinney <wesmckinn at gmail.com> wrote:
>> >>> >> >>
>> >>> >> >> @Andy
>> >>> >> >>
>> >>> >> >> pydata at googlegroups.com has 2734 members. Based on recent traffic it
>> >>> >> >> is really a user / Q&A mailing list, not a place for the
>> >>> >> >> maintainers/steering committees of major projects to speak publicly
>> >>> >> >> with one another (where discussions are public, archived, searchable).
>> >>> >> >> I have observed that there are many discussions happening between the
>> >>> >> >> developers of projects on an ad hoc basis and on ad hoc communication
>> >>> >> >> channels (both private and public). Partly there is no obvious place
>> >>> >> >> for, e.g., the developers of pandas and dask to have a public
>> >>> >> >> discussion, that is not necessarily "partisan" to one of those
>> >>> >> >> projects.
>> >>> >> >>
>> >>> >> >> As another example issue, there is not an obvious place to raise
>> >>> >> >> issues. Back in the day I think numpy-discussion or scipy-user used to
>> >>> >> >> partly serve this purpose, but the centers of gravity have shifted.
>> >>> >> >>
>> >>> >> >> - Wes
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> On Wed, Dec 26, 2018 at 9:41 PM Andy Ray Terrel <andy.terrel at gmail.com> wrote:
>> >>> >> >> >
>> >>> >> >> > I'm not completely clear what is being asked for since pydata at googlegroups.com already exists. Since NumFOCUS is promoting the PyData conference and helping build the brand for the ecosystem, I wonder if a home like pydata-dev at numfocus.org would be interesting for folks?
>> >>> >> >> >
>> >>> >> >> > It is also my understanding that there will be a fuller steering committee set up for the conferences next year. I propose we do the same for the technical structure. As is, I manage the website and github repos but there is not much dictating how I manage these.
>> >>> >> >> >
>> >>> >> >> > -- Andy
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> > On Wed, Dec 26, 2018 at 6:00 PM Nathaniel Smith <njs at pobox.com> wrote:
>> >>> >> >> >>
>> >>> >> >> >> Other examples of discourse used for dev discussion include:
>> >>> >> >> >>
>> >>> >> >> >> - https://internals.rust-lang.org/ -- main dev forum for rust
>> >>> >> >> >> - https://discuss.python.org/ -- potential replacement for
>> >>> >> >> >> python-{committers,dev,users}, still experimental but where a ton of
>> >>> >> >> >> the python governance discussion happened
>> >>> >> >> >>
>> >>> >> >> >> My impression so far is that discourse takes a bit of adjustment
>> >>> >> >> >> compared to mailing lists, but it has a lot of valuable features like
>> >>> >> >> >> multi-quoting, markdown (code blocks, links, ...), solid moderation
>> >>> >> >> >> tools (e.g. if a discussion diverges you can retroactively split parts
>> >>> >> >> >> of it out into a new topic), polls (these were incredibly useful for
>> >>> >> >> >> taking the temperature of the community during the governance
>> >>> >> >> >> discussions), ability to reply to messages that were posted before you
>> >>> >> >> >> joined the list, configurable notifications (email me everything /
>> >>> >> >> >> email me when a new topic is created / email me a summary weekly /
>> >>> >> >> >> ...), ...
>> >>> >> >> >>
>> >>> >> >> >> -n
>> >>> >> >> >>
>> >>> >> >> >> On Wed, Dec 26, 2018 at 3:41 PM Matthew Rocklin <mrocklin at gmail.com> wrote:
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >> >>> Copying the mailing list
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >> > Whoops!  E-mail fail on my part.
>> >>> >> >> >> >
>> >>> >> >> >> >>> Discourse is interesting. It seems to be used (at least in PyTorch's
>> >>> >> >> >> >>> case) as more of a modern message board for users than a place for
>> >>> >> >> >> >>> long-form discussions between project developers.
>> >>> >> >> >> >>>
>> >>> >> >> >> >>> IMHO having a cross-project developer mailing list is probably overdue
>> >>> >> >> >> >>> -- I think we can do a better job the next couple of years
>> >>> >> >> >> >>> coordinating (colluding?) with each other. A lot of coordination does
>> >>> >> >> >> >>> of course in private, project-level, or other ad-hoc basis. It would
>> >>> >> >> >> >>> help to be able to discuss ecosystem-level problems and possible
>> >>> >> >> >> >>> solutions.
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >> > Entirely agreed.  And I think that an e-mail list is the obvious first choice here.
>> >>> >> >> >> >
>> >>> >> >> >> > I'm bringing up discourse as an alternative for consideration.  This is for a couple reasons:
>> >>> >> >> >> >
>> >>> >> >> >> > I'm slightly concerned that a broad ranging e-mail list that encompasses all of PyData would get noisy.  For example I can imagine lengthy conversations on visualization or probabalistic programming that, while I find important, would likely want to take a pass on.  Having a service that includes tags and subscription to those tags may have value.
>> >>> >> >> >> > E-mail list archives tend to collect dust.  If we're doing long-range cross-project coordination then those conversations might have long term value.  We might want to cross reference them, upvote them, subscribe to them, and so on.
>> >>> >> >> >> >
>> >>> >> >> >> > In regards to PyTorch's discuss in particular I agree that it is used more as a user forum, which I agree is a different use case than what Wes is proposing here.  I mostly pointed to it so that people could get a sense of what an active system looks like.
>> >>> >> >> >> >
>> >>> >> >> >> > Regardless, I encourage this conversation to happen with a broader set of people.  I believe that other groups are considering these topics as well and may have thoughts beyond those that have been expressed here.  I'm not sure how best to bootstrap this process, other than an e-mail to maybe the NumFOCUS mailing list and perhaps a tweet?
>> >>> >> >> >> >
>> >>> >> >> >> > > There's both a NumFOCUS discourse and zulip, I believe, but neither is particularly active. Whether those should be considered possible starting points or cautionary tales I leave to y'all.
>> >>> >> >> >> >
>> >>> >> >> >> > Yeah, I should also amend my previous statement from "how about discourse?" to "is there anything more appropriate than an e-mail list?".  Discourse is the service around which I've seen the most activity recently but I agree that in our community, it hasn't really taken off that well.
>> >>> >> >> >> >
>> >>> >> >> >> > And just to reiterate, I think that an e-mail list would be great.  Just wanted to throw out some other thoughts.
>> >>> >> >> >> >
>> >>> >> >> >> > Best,
>> >>> >> >> >> > -matt
>> >>> >> >> >> >>>
>> >>> >> >> >> >>>
>> >>> >> >> >> >>> > On Wed, Dec 26, 2018 at 8:47 AM Wes McKinney <wesmckinn at gmail.com> wrote:
>> >>> >> >> >> >>> >>
>> >>> >> >> >> >>> >> I sent a request to postmaster @ python.o to create
>> >>> >> >> >> >>> >> pydata-dev at python.org. We can also use google groups if others prefer
>> >>> >> >> >> >>> >> that
>> >>> >> >> >> >>> >>
>> >>> >> >> >> >>> >> On Tue, Dec 25, 2018 at 3:59 PM Joris Van den Bossche
>> >>> >> >> >> >>> >> <jorisvandenbossche at gmail.com> wrote:
>> >>> >> >> >> >>> >> >
>> >>> >> >> >> >>> >> > Giving the growing ecosysten of data tools (in some way related to pandas, but not pandas itself), I am also +1 on such a list. I think that would be welcome, and not aware of anything existing.
>> >>> >> >> >> >>> >> >
>> >>> >> >> >> >>> >> > Joris
>> >>> >> >> >> >>> >> >
>> >>> >> >> >> >>> >> > Op di 25 dec. 2018 02:19 schreef Stephan Hoyer <shoyer at gmail.com:
>> >>> >> >> >> >>> >> >>
>> >>> >> >> >> >>> >> >> +1 for pydata-dev
>> >>> >> >> >> >>> >> >>
>> >>> >> >> >> >>> >> >> I don't think there's a list quite like this today.
>> >>> >> >> >> >>> >> >>
>> >>> >> >> >> >>> >> >> On Mon, Dec 24, 2018 at 6:11 PM Wes McKinney <wesmckinn at gmail.com> wrote:
>> >>> >> >> >> >>> >> >>>
>> >>> >> >> >> >>> >> >>> I'm talking about public archived communication channels
>> >>> >> >> >> >>> >> >>>
>> >>> >> >> >> >>> >> >>> On Mon, Dec 24, 2018, 7:57 PM William Ayd <william.ayd at icloud.com wrote:
>> >>> >> >> >> >>> >> >>>>
>> >>> >> >> >> >>> >> >>>> What do you think is missing from the existing PyData conferences? I’ve only been to the one in LA but it seemed to be somewhat in the direction of what you are asking for.
>> >>> >> >> >> >>> >> >>>>
>> >>> >> >> >> >>> >> >>>> > On Dec 24, 2018, at 3:02 PM, Wes McKinney <wesmckinn at gmail.com> wrote:
>> >>> >> >> >> >>> >> >>>> >
>> >>> >> >> >> >>> >> >>>> > hi folks,
>> >>> >> >> >> >>> >> >>>> >
>> >>> >> >> >> >>> >> >>>> > Prompted by some recent discussions I wondered what you all think
>> >>> >> >> >> >>> >> >>>> > would be the best venue to have public discussions that involve other
>> >>> >> >> >> >>> >> >>>> > open source projects that are generally 1 degree of separation away
>> >>> >> >> >> >>> >> >>>> > from pandas. Sort of like "pydata-dev", or something. Is there
>> >>> >> >> >> >>> >> >>>> > something like this already that I just missed?
>> >>> >> >> >> >>> >> >>>> >
>> >>> >> >> >> >>> >> >>>> > As context, I'm trying to travel less and go to fewer conferences the
>> >>> >> >> >> >>> >> >>>> > next couple of years, and spend more time coding and writing, but I
>> >>> >> >> >> >>> >> >>>> > still want to talk with people (asynchronously) about things, and
>> >>> >> >> >> >>> >> >>>> > preferably in public.
>> >>> >> >> >> >>> >> >>>> >
>> >>> >> >> >> >>> >> >>>> > - Wes
>> >>> >> >> >> >>> >> >>>> > _______________________________________________
>> >>> >> >> >> >>> >> >>>> > Pandas-dev mailing list
>> >>> >> >> >> >>> >> >>>> > Pandas-dev at python.org
>> >>> >> >> >> >>> >> >>>> > https://mail.python.org/mailman/listinfo/pandas-dev
>> >>> >> >> >> >>> >> >>>>
>> >>> >> >> >> >>> >> >>> _______________________________________________
>> >>> >> >> >> >>> >> >>> Pandas-dev mailing list
>> >>> >> >> >> >>> >> >>> Pandas-dev at python.org
>> >>> >> >> >> >>> >> >>> https://mail.python.org/mailman/listinfo/pandas-dev
>> >>> >> >> >> >>> >> >>
>> >>> >> >> >> >>> >> >> _______________________________________________
>> >>> >> >> >> >>> >> >> Pandas-dev mailing list
>> >>> >> >> >> >>> >> >> Pandas-dev at python.org
>> >>> >> >> >> >>> >> >> https://mail.python.org/mailman/listinfo/pandas-dev
>> >>> >> >> >> >>> >> _______________________________________________
>> >>> >> >> >> >>> >> Pandas-dev mailing list
>> >>> >> >> >> >>> >> Pandas-dev at python.org
>> >>> >> >> >> >>> >> https://mail.python.org/mailman/listinfo/pandas-dev
>> >>> >> >> >> >>> _______________________________________________
>> >>> >> >> >> >>> Pandas-dev mailing list
>> >>> >> >> >> >>> Pandas-dev at python.org
>> >>> >> >> >> >>> https://mail.python.org/mailman/listinfo/pandas-dev
>> >>> >> >> >> >
>> >>> >> >> >> > _______________________________________________
>> >>> >> >> >> > Pandas-dev mailing list
>> >>> >> >> >> > Pandas-dev at python.org
>> >>> >> >> >> > https://mail.python.org/mailman/listinfo/pandas-dev
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >> --
>> >>> >> >> >> Nathaniel J. Smith -- https://vorpus.org
>> >>> >> >> >> _______________________________________________
>> >>> >> >> >> Pandas-dev mailing list
>> >>> >> >> >> Pandas-dev at python.org
>> >>> >> >> >> https://mail.python.org/mailman/listinfo/pandas-dev
>> >>> >> >> >
>> >>> >> >> > _______________________________________________
>> >>> >> >> > Pandas-dev mailing list
>> >>> >> >> > Pandas-dev at python.org
>> >>> >> >> > https://mail.python.org/mailman/listinfo/pandas-dev
>> >>> >> _______________________________________________
>> >>> >> Pandas-dev mailing list
>> >>> >> Pandas-dev at python.org
>> >>> >> https://mail.python.org/mailman/listinfo/pandas-dev