[Pandas-dev] Discourse discussion forum

[Marc Garcia]

We had a discussion in the NumFOCUS summit about Discourse. People with
more experience also considered that is not feasible to use Discourse for
all projects in a reasonable and organized way. So, if nobody objects we'll
move forward with the pandas Discourse.

I think a PyData / NumFOCUS one for communication among projects can make
sense. If anybody have ideas on categories we should have, what to set up
to communicate among projects... They are more than welcome. Otherwise
we'll keep adding figure out what works best as we use Discourse.

If you want to start signing up in our Discourse, pandas.discourse.group, I
think we can start using it for discussions, and delete this list once
we're confident. There are already few people in the core team that is in
Discourse and got admin rights. Please let us know if you sign up, so we
can grant you admin rights too.

Cheers!

On Tue, Oct 29, 2019 at 12:45 PM Andy Ray Terrel <andy at numfocus.org> wrote:

> Okay well I can go bug the heads of all the pydata projects, but the
> confusion comes when a user doesn't know where to post. Having lots of
> discourse sites, seems like it will lead to confusion and more work on
> maintainers to curate the community discussion.
>
>
> On Tue, Oct 29, 2019 at 10:21 AM Marc Garcia <garcia.marc at gmail.com>
> wrote:
>
>> That's a good point. I guess it doesn't make a big difference in terms of
>> organization of the threads, as a discussion on something dask-pandas will
>> still need to be in one of the categories (pandas-dev or dask-dev). But
>> being able to tag people from other projects could be useful.
>>
>> But I still think that having separate discourse instances will make our
>> lives easier. Feels like a huge mess to have all projects in the same
>> instance with the navigation of discourse.
>>
>> On Tue, Oct 29, 2019 at 12:09 PM Andy Ray Terrel <andy at numfocus.org>
>> wrote:
>>
>>> I think the value many have is for cross project issues, but maybe those
>>> are few and far between.
>>>
>>> On Tue, Oct 29, 2019 at 10:07 AM Marc Garcia <garcia.marc at gmail.com>
>>> wrote:
>>>
>>>> I personally don't see the value of having a common discourse for all
>>>> the projects, where the top-level is a list of possibly 100 items, where
>>>> pandas has few groups lost there, and not more structure than that, as
>>>> opposed to have a discourse per project.
>>>>
>>>> Single-login is the only advantage I can see, and this can also be
>>>> achieved with separate groups for what I've seen.
>>>>
>>>> Tom, Joris, I think you were the ones who preferred having a common
>>>> discourse. Does it still sounds as the best option, given the limitations?
>>>>
>>>> On Tue, Oct 29, 2019 at 11:51 AM Andy Ray Terrel <andy at numfocus.org>
>>>> wrote:
>>>>
>>>>> Sorry I've been traveling.
>>>>>
>>>>> I have https://pydata.discourse. <http://pydata.discourse.org>group set
>>>>> up. I can send out invites.
>>>>>
>>>>> I guess as you have pointed out, we can set up categories for each
>>>>> project, e.g. dask-users, pandas-users, pandas-dev, but maybe not exactly
>>>>> what you want.
>>>>>
>>>>> Happy to invite anyone to the discourse instance before we open it up
>>>>> to the wild
>>>>>
>>>>> -- Andy
>>>>>
>>>>> On Tue, Oct 29, 2019 at 9:24 AM Marc Garcia <garcia.marc at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Andy, could you experiment on having multiple projects in a single
>>>>>> discourse? I saw the PyData one was activated some time ago.
>>>>>>
>>>>>> If it doesn't look feasible as I think, let me know so I'll move
>>>>>> forward discussing what to have in the pandas one.
>>>>>>
>>>>>> Cheers!
>>>>>>
>>>>>> On Wed, Sep 25, 2019 at 8:03 AM Marc Garcia <garcia.marc at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Discourse has private categories, we already have a private
>>>>>>> "Maintainers" one, that only admins can see and use. And there are other
>>>>>>> permissions levels that can be used. For example, we can have a private
>>>>>>> category for the memebers of the code of conduct committee... I just need
>>>>>>> to check if we can associate email addresses to those groups, so when
>>>>>>> someone emails to coc at pandas.io the messages are posted in that
>>>>>>> private group. But if we can set up that as we need, I think we should be
>>>>>>> able to replace all those and centralize everything in Discourse.
>>>>>>>
>>>>>>> I'm skeptical on being able to set up a global Discourse for all the
>>>>>>> ecosystem, where things are easy to find, based on how Discourse works and
>>>>>>> the tests I did. I'd move forward with our own for now if nobody is able to
>>>>>>> set that up.
>>>>>>>
>>>>>>> Andy, I got the pandas account approved in minutes. I see that we
>>>>>>> can have a custom domain, so you can use the pandas and see if we can
>>>>>>> manage to have multiple projects in a way we like, and if we do we just
>>>>>>> change the domain to discuss.pydata.org (or whatever). You're
>>>>>>> already an admin, feel free to experiment and change the set up as you need.
>>>>>>>
>>>>>>> Maarten, not sure I understand your point. Not a fan of Discourse so
>>>>>>> far, but I think having the user and the devs discussions in a single place
>>>>>>> makes it easier to find the information, and I think Discourse interface
>>>>>>> also makes it easier to find compared to mailman, or google groups.
>>>>>>> Regardless of gitter (there are no important discussions or decision making
>>>>>>> there I think), would you prefer to stay with mailman and google groups
>>>>>>> over Discourse? Or what you think would be the ideal or best option?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> On Wed, Sep 25, 2019 at 8:39 AM Joris Van den Bossche <
>>>>>>> jorisvandenbossche at gmail.com> wrote:
>>>>>>>
>>>>>>>> What do other people think about starting to use discourse for
>>>>>>>> pandas?
>>>>>>>> (and about sharing it with other projects or having our own?)
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> On the existing lists: I don't think discourse would replace the
>>>>>>>> core devs list (that is intentionally private). And IMO also not gitter
>>>>>>>> (discourse is not a real-time chat).
>>>>>>>>
>>>>>>>> Joris
>>>>>>>>
>>>>>>>> On Fri, 20 Sep 2019 at 14:58, Marc Garcia <garcia.marc at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> For what I've seen I'd say that Discourse can be configured to
>>>>>>>>> interact with a category like a distribution list (subscribe and have an
>>>>>>>>> email address to send messages there). Not sure, but for the settings I've
>>>>>>>>> seen should be possible.
>>>>>>>>>
>>>>>>>>> Personally I think it should replace all the existing lists:
>>>>>>>>> - pydata google group
>>>>>>>>> - pandas-dev (this)
>>>>>>>>> - core devs list
>>>>>>>>>
>>>>>>>>> I'm also ok to get rid of gitter once we move to discourse (also
>>>>>>>>> ok to keep it if people find it useful, but I rarely use it).
>>>>>>>>>
>>>>>>>>> I created an issue for this discussion some time ago:
>>>>>>>>> https://github.com/pandas-dev/pandas/issues/27903
>>>>>>>>>
>>>>>>>>> On Fri, Sep 20, 2019 at 1:50 PM Tom Augspurger <
>>>>>>>>> tom.augspurger88 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 20, 2019 at 6:57 AM Andy Terrel <andy at numfocus.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks Joris for splitting the thread, sorry if I hijacked the
>>>>>>>>>>> other one.
>>>>>>>>>>>
>>>>>>>>>>> For some discussion from numpy you can see here
>>>>>>>>>>> https://github.com/numpy/numpy.org/issues/28
>>>>>>>>>>>
>>>>>>>>>>> Julia and Jupyter both run their own discourse but Dask, Numpy,
>>>>>>>>>>> Scipy have all told me “I don’t want to run it ourselves but be part of a
>>>>>>>>>>> larger one”
>>>>>>>>>>>
>>>>>>>>>>> I bet we can figure out how to organize it.
>>>>>>>>>>>
>>>>>>>>>>> I just put in an application to get pydata.discourse.org.
>>>>>>>>>>>
>>>>>>>>>>> — Andy
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Sep 20, 2019 at 6:52 AM Joris Van den Bossche <
>>>>>>>>>>> jorisvandenbossche at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> (let's use a new thread for discourse, as it is a different
>>>>>>>>>>>> discussion from the website hosting I think, regardless whether OVH might
>>>>>>>>>>>> also host discourse)
>>>>>>>>>>>>
>>>>>>>>>>>> I am not familiar enough myself with discourse to know whether
>>>>>>>>>>>> multiple projects sharing a single discourse will become annoying. But
>>>>>>>>>>>> indeed, that sounds as it needs some kind of hierarchical category /
>>>>>>>>>>>> tagging.
>>>>>>>>>>>>
>>>>>>>>>>>> For pandas itself: I think I quite like the idea of having a
>>>>>>>>>>>> discourse, but *if* we do that, we should think about how that
>>>>>>>>>>>> fits with / replaces / adds to /... some of the other communication
>>>>>>>>>>>> channels (pandas-dev mailing list, pydata mailing list, github issues, ..).
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> IMO, we can replace the pandas-dev & pydata mailing lists with
>>>>>>>>>> it. Possibly gitter as well.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Joris
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, 20 Sep 2019 at 13:18, Marc Garcia <
>>>>>>>>>>>> garcia.marc at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I'm fine with that conceptually, but I think Discourse will
>>>>>>>>>>>>> make things quite tricky to find things then.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We already got our discourse approved, if you want to join it
>>>>>>>>>>>>> an experiment with the setting. But it's the first thing I tried, and after
>>>>>>>>>>>>> you join a category (project), everything feels like it's in the same place
>>>>>>>>>>>>> (even if subcategories and tags exist). And I think we need at least a
>>>>>>>>>>>>> clear separation between pandas/users pandas/contributors discussions.
>>>>>>>>>>>>>
>>>>>>>>>>>>> May be I just couldn't find the settings, let me know if you
>>>>>>>>>>>>> manage to get a multi-project set up that makes sense.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Sep 20, 2019 at 12:07 PM Tom Augspurger <
>>>>>>>>>>>>> tom.augspurger88 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'd prefer to join a discourse along with NumPy, Dask, and
>>>>>>>>>>>>>> other PyData or NumFOCUS projects, rather than going out on our own.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Sep 20, 2019 at 4:47 AM Marc Garcia <
>>>>>>>>>>>>>> garcia.marc at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I don't know much about discourse, but why do we want to
>>>>>>>>>>>>>>> self-host it? Seems like Discourse does it for free for open source
>>>>>>>>>>>>>>> projects: https://free.discourse.group/ And I don't think
>>>>>>>>>>>>>>> we want another system to maintain. Am I missing something?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I applied for https://pandas.discourse.group, so we can
>>>>>>>>>>>>>>> give it a try. We should have it approved and working in couple of days.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> For what I saw, Discourse has one level of categories, so I
>>>>>>>>>>>>>>> guess we want one per project, so we can have categories for "Users",
>>>>>>>>>>>>>>> "Contributors", "Ecosystem"... or something similar. I guess if we have a
>>>>>>>>>>>>>>> single Discourse for NumFOCUS, every project will be a category, and it'll
>>>>>>>>>>>>>>> be difficult to group conversations.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If anyone already has experience with Discourse and
>>>>>>>>>>>>>>> disagrees with my guesses, please let me know.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Sep 18, 2019 at 4:32 PM Andy Terrel <
>>>>>>>>>>>>>>> andy at numfocus.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sounds great to me. Just let me know where everything goes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> NumPy wants me to help host a discourse for them, maybe OVH
>>>>>>>>>>>>>>>> would be a good place to do that as well, (although I would be more
>>>>>>>>>>>>>>>> inclinded if it was pydata and we had pandas, scipy, and numpy on it).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -- Andy
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Sep 18, 2019 at 8:51 AM Tom Augspurger <
>>>>>>>>>>>>>>>> tom.augspurger88 at gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Sounds good w.r.t crediting OVH on those pages.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> For the ASV results at pandas.pydata.org/speed (which I
>>>>>>>>>>>>>>>>> now notice is currently broken for pandas), the only thing on the webserver
>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>> cron job doing a `git pull` from
>>>>>>>>>>>>>>>>> https://github.com/asv-runner/asv-collection, from within
>>>>>>>>>>>>>>>>> `/usr/share/nginx`.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Sep 18, 2019 at 8:18 AM Marc Garcia <
>>>>>>>>>>>>>>>>> garcia.marc at gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> An update on the new website infrastructure. We need to
>>>>>>>>>>>>>>>>>> finish discussing the details, but OVH is happy to provide the hosting for
>>>>>>>>>>>>>>>>>> the pandas infrastructure we need.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> My initial idea is to credit them in the page with the
>>>>>>>>>>>>>>>>>> rest of the sponsors in the new website:
>>>>>>>>>>>>>>>>>> https://datapythonista.github.io/pandas-web/community/team.html#institutional-partners and
>>>>>>>>>>>>>>>>>> also in the top right corner of the runnable code widgets (see for example
>>>>>>>>>>>>>>>>>> where Binder is credited here: https://spacy.io/).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> What I'd like to ask is:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. For the production website and docs (static content
>>>>>>>>>>>>>>>>>> only, for the traffic we need):
>>>>>>>>>>>>>>>>>> https://us.ovhcloud.com/products/public-cloud/object-storage
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2. For our tools and processes, like the benchmarks,
>>>>>>>>>>>>>>>>>> builds, CI stuff (temporary publish the docs for every PR,...):
>>>>>>>>>>>>>>>>>> https://www.ovh.co.uk/vps/vps-ssd.xml (VPS SSD 3)
>>>>>>>>>>>>>>>>>> 3. For BinderHub (runnable code in our docs, launch
>>>>>>>>>>>>>>>>>> tutorials on Binder...):
>>>>>>>>>>>>>>>>>> https://www.ovh.co.uk/public-cloud/kubernetes/
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> For the BinderHub, QuantStack offered help with the set
>>>>>>>>>>>>>>>>>> up (which is great, because I don't know much about Binder myself, and I'm
>>>>>>>>>>>>>>>>>> not sure if anyone else does or wants to take care of this). I don't think
>>>>>>>>>>>>>>>>>> it'll be easy to estimate how big is the cluster we need beforehand, but I
>>>>>>>>>>>>>>>>>> guess we can add things to Binder iteratively, and have more info as we
>>>>>>>>>>>>>>>>>> grow.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> OVH gave us a 200 euros voucher to experiment with the
>>>>>>>>>>>>>>>>>> different services. Let me know how all this sounds, and if there are no
>>>>>>>>>>>>>>>>>> objections, I'll create an account and buy those services with the voucher,
>>>>>>>>>>>>>>>>>> and I'll start to prototype and see how everything works.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Aug 20, 2019 at 11:06 PM Marc Garcia <
>>>>>>>>>>>>>>>>>> garcia.marc at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Somehow related to the work on the new website (
>>>>>>>>>>>>>>>>>>> https://github.com/pandas-dev/pandas/pull/28014), I've
>>>>>>>>>>>>>>>>>>> been discussing with the Binder team, and looks like should be quite easy
>>>>>>>>>>>>>>>>>>> soon (with a Sphinx extension) to make all the documentation pages runnable
>>>>>>>>>>>>>>>>>>> with Binder, directly from the website (without opening the page as a
>>>>>>>>>>>>>>>>>>> Jupyter in mybinder).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> While they are very happy with the idea of having this
>>>>>>>>>>>>>>>>>>> is pandas, it's uncertain if the current infrastructure Binder has got, is
>>>>>>>>>>>>>>>>>>> able to handle all the traffic we would send. And scikit-learn is working
>>>>>>>>>>>>>>>>>>> on it too (today they added to the dev docs a link to mybinder to run the
>>>>>>>>>>>>>>>>>>> examples).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I'm discussing with OVH (their infrastructure provider)
>>>>>>>>>>>>>>>>>>> on whether they'd be happy to provide a dedicated BinderHub specific to
>>>>>>>>>>>>>>>>>>> pandas (or may be we can have one for all NumFOCUS projects). We'll see how
>>>>>>>>>>>>>>>>>>> it goes, but wanted to let you know, so you're updated, and in case anyone
>>>>>>>>>>>>>>>>>>> is interested in participating in the discussions. Of course before any
>>>>>>>>>>>>>>>>>>> decision is made I'll open a discussion here or on GitHub.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> As part of the discussion I'm also trying to get a
>>>>>>>>>>>>>>>>>>> server for the website, and one for development stuff. Specfically for the
>>>>>>>>>>>>>>>>>>> dev docs (including rendered docs of every PR) and the GitHub app that will
>>>>>>>>>>>>>>>>>>> generate them. I guess it should be very easy to find a sponsor for these
>>>>>>>>>>>>>>>>>>> two servers (in exchange of a small note in the footer of the website, or
>>>>>>>>>>>>>>>>>>> something like that).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Let me know if you have any comment, want to be involved
>>>>>>>>>>>>>>>>>>> or whatever.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>> Pandas-dev mailing list
>>>>>>>>>>>>>>>>>> Pandas-dev at python.org
>>>>>>>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Andy R. Terrel, PhD
>>>>>>>>>>>>>>>> President
>>>>>>>>>>>>>>>> NumFOCUS
>>>>>>>>>>>>>>>> andy at numfocus.org
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Pandas-dev mailing list
>>>>>>>>>>>>> Pandas-dev at python.org
>>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>> Andy R. Terrel, PhD
>>>>>>>>>>> President
>>>>>>>>>>> NumFOCUS
>>>>>>>>>>> andy at numfocus.org
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>> Pandas-dev mailing list
>>>>>>>> Pandas-dev at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>> Pandas-dev mailing list
>>>>>> Pandas-dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Andy R. Terrel, PhD
>>>>> President, NumFOCUS
>>>>> andy at numfocus.org
>>>>>
>>>> _______________________________________________
>>>> Pandas-dev mailing list
>>>> Pandas-dev at python.org
>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>
>>>
>>>
>>> --
>>> Andy R. Terrel, PhD
>>> President, NumFOCUS
>>> andy at numfocus.org
>>>
>> _______________________________________________
>> Pandas-dev mailing list
>> Pandas-dev at python.org
>> https://mail.python.org/mailman/listinfo/pandas-dev
>>
>
>
> --
> Andy R. Terrel, PhD
> President, NumFOCUS
> andy at numfocus.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pandas-dev/attachments/20191104/3da07abe/attachment-0001.html>